{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/densely-connected-normalizing-flows","title":"Densely connected normalizing flows","arxiv_id":"2106.04627","date":"2021-06-08","proceeding":"NeurIPS 2021 12","authors":["Matej Grcić","Ivan Grubišić","Siniša Šegvić"],"abstract":"Normalizing flows are bijective mappings between inputs and latent representations with a fully factorized distribution. They are very attractive due to exact likelihood valuation and efficient sampling. However, their effective capacity is often insufficient since the bijectivity constraint limits the model width. We address this issue by incrementally padding intermediate representations with noise. We precondition the noise in accordance with previous invertible units, which we describe as cross-unit coupling. Our invertible glow-like modules increase the model expressivity by fusing a densely connected block with Nystrom self-attention. We refer to our architecture as DenseFlow since both cross-unit and intra-module couplings rely on dense connectivity. Experiments show significant improvements due to the proposed contributions and reveal state-of-the-art density estimation under moderate computing budgets.","url_abs":"https://arxiv.org/abs/2106.04627v3","url_pdf":"https://arxiv.org/pdf/2106.04627v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"densely-connected-normalizing-flows","repo_url":"https://github.com/matejgrcic/DenseFlow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-2.0"}},{"paper_slug":"densely-connected-normalizing-flows","repo_url":"https://github.com/MindSpore-scientific-2/code-11/tree/main/nystrom-attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"densely-connected-normalizing-flows","repo_url":"https://github.com/MindSpore-scientific-2/code-8/tree/main/nystrom-attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"densely-connected-normalizing-flows","repo_url":"https://github.com/MindSpore-scientific/code-13/tree/main/nystrom-attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"affine-coupling","method_name":"Affine Coupling"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-cifar-10","task":"Image Generation","dataset":"CIFAR-10","model":"DenseFlow-74-10","rank_in_archive_order":71,"of":78,"metrics":{"FID":"34.90"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-64x64","task":"Image Generation","dataset":"CelebA 64x64","model":"DenseFlow-74-10","rank_in_archive_order":39,"of":39,"metrics":{"bits/dimension":"1.99"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"DenseFlow-74-10","rank_in_archive_order":16,"of":35,"metrics":{"bpd":"3.63"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-64x64","task":"Image Generation","dataset":"ImageNet 64x64","model":"DenseFlow-74-10","rank_in_archive_order":38,"of":65,"metrics":{"Bits per dim":"3.35 (different downsampling)"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.04627","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}