{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/normalizing-flows-are-capable-generative","title":"Normalizing Flows are Capable Generative Models","arxiv_id":"2412.06329","date":"2024-12-09","proceeding":null,"authors":["Shuangfei Zhai","Ruixiang Zhang","Preetum Nakkiran","David Berthelot","Jiatao Gu","Huangjie Zheng","Tianrong Chen","Miguel Angel Bautista","Navdeep Jaitly","Josh Susskind"],"abstract":"Normalizing Flows (NFs) are likelihood-based models for continuous inputs. They have demonstrated promising results on both density estimation and generative modeling tasks, but have received relatively little attention in recent years. In this work, we demonstrate that NFs are more powerful than previously believed. We present TarFlow: a simple and scalable architecture that enables highly performant NF models. TarFlow can be thought of as a Transformer-based variant of Masked Autoregressive Flows (MAFs): it consists of a stack of autoregressive Transformer blocks on image patches, alternating the autoregression direction between layers. TarFlow is straightforward to train end-to-end, and capable of directly modeling and generating pixels. We also propose three key techniques to improve sample quality: Gaussian noise augmentation during training, a post training denoising procedure, and an effective guidance method for both class-conditional and unconditional settings. Putting these together, TarFlow sets new state-of-the-art results on likelihood estimation for images, beating the previous best methods by a large margin, and generates samples with quality and diversity comparable to diffusion models, for the first time with a stand-alone NF model. We make our code available at https://github.com/apple/ml-tarflow.","url_abs":"https://arxiv.org/abs/2412.06329v2","url_pdf":"https://arxiv.org/pdf/2412.06329v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"normalizing-flows-are-capable-generative","repo_url":"https://github.com/apple/ml-tarflow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"normalizing-flows-are-capable-generative","repo_url":"https://github.com/encoreus/gs-jacobi_for_tarflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"normalizing-flows-are-capable-generative","repo_url":"https://github.com/MindSpore-scientific-2/code-10/tree/main/temporal-normalizing-flows","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"unconditional-image-generation","task_name":"Unconditional Image Generation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-imagenet-128x128","task":"Image Generation","dataset":"ImageNet 128x128","model":"TarFlow","rank_in_archive_order":12,"of":23,"metrics":{"FID":"5.03"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-64x64","task":"Image Generation","dataset":"ImageNet 64x64","model":"TarFlow","rank_in_archive_order":20,"of":65,"metrics":{"Bits per dim":"2.99","FID":"2.9"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.06329","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.06329"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/apple/mltarflow","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-10/tree/main/temporal-normalizing-flows","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/apple/ml-tarflow","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/encoreus/gs-jacobi_for_tarflow","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"484454e1a5aea143","entry":"gaussian_log_prob","repo":"apple/ml-tarflow","repo_kind":"official","path":"evaluate_bpd.py","file_url":"https://github.com/apple/ml-tarflow/blob/HEAD/evaluate_bpd.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"484454e1a5aea143"}},{"code_sha256_prefix":"16298c972234921e","entry":"generate_sample","repo":"MindSpore-scientific-2/code-10","repo_kind":"listed","path":"QuantExplainNLP_AlgoAnalysis/data_preprocessing.py","file_url":"https://github.com/MindSpore-scientific-2/code-10/blob/HEAD/QuantExplainNLP_AlgoAnalysis/data_preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16298c972234921e"}},{"code_sha256_prefix":"5919a74fc358ee01","entry":"get_feature_names","repo":"MindSpore-scientific-2/code-10","repo_kind":"listed","path":"QuantExplainNLP_AlgoAnalysis/explainability.py","file_url":"https://github.com/MindSpore-scientific-2/code-10/blob/HEAD/QuantExplainNLP_AlgoAnalysis/explainability.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5919a74fc358ee01"}},{"code_sha256_prefix":"ef93260b5137f73b","entry":"maybe_num_nodes","repo":"MindSpore-scientific-2/code-10","repo_kind":"listed","path":"RGNN/model_simple.py","file_url":"https://github.com/MindSpore-scientific-2/code-10/blob/HEAD/RGNN/model_simple.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef93260b5137f73b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}