{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-flow-networks","title":"Bayesian Flow Networks","arxiv_id":"2308.07037","date":"2023-08-14","proceeding":null,"authors":["Alex Graves","Rupesh Kumar Srivastava","Timothy Atkinson","Faustino Gomez"],"abstract":"This paper introduces Bayesian Flow Networks (BFNs), a new class of generative model in which the parameters of a set of independent distributions are modified with Bayesian inference in the light of noisy data samples, then passed as input to a neural network that outputs a second, interdependent distribution. Starting from a simple prior and iteratively updating the two distributions yields a generative procedure similar to the reverse process of diffusion models; however it is conceptually simpler in that no forward process is required. Discrete and continuous-time loss functions are derived for continuous, discretised and discrete data, along with sample generation procedures. Notably, the network inputs for discrete data lie on the probability simplex, and are therefore natively differentiable, paving the way for gradient-based sample guidance and few-step generation in discrete domains such as language modelling. The loss function directly optimises data compression and places no restrictions on the network architecture. In our experiments BFNs achieve competitive log-likelihoods for image modelling on dynamically binarized MNIST and CIFAR-10, and outperform all known discrete diffusion models on the text8 character-level language modelling task.","url_abs":"https://arxiv.org/abs/2308.07037v5","url_pdf":"https://arxiv.org/pdf/2308.07037v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-flow-networks","repo_url":"https://github.com/nnaisense/bayesian-flow-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"data-compression","task_name":"Data Compression"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-binarized-mnist","task":"Image Generation","dataset":"Binarized MNIST","model":"BFN","rank_in_archive_order":3,"of":10,"metrics":{"nats":"77.87"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-text8","task":"Language Modelling","dataset":"Text8","model":"BFN","rank_in_archive_order":22,"of":24,"metrics":{"Bit per Character (BPC)":"1.41"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.07037","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.07037"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nnaisense/bayesian-flow-networks","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":11,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c362b4cbfac8170f","entry":"attention_inner_heads","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_vdm.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_vdm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c362b4cbfac8170f"}},{"code_sha256_prefix":"0af60baeeb1fb185","entry":"bin_mnist_cts_transform","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"data.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0af60baeeb1fb185"}},{"code_sha256_prefix":"2ae69bceb65a76e8","entry":"bin_mnist_transform","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"data.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ae69bceb65a76e8"}},{"code_sha256_prefix":"b17ebf2b2ef1cb60","entry":"gelu","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/transformer.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/transformer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b17ebf2b2ef1cb60"}},{"code_sha256_prefix":"179411226b633de6","entry":"get_timestep_embedding","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_vdm.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_vdm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"179411226b633de6"}},{"code_sha256_prefix":"3441023d420317ab","entry":"init_checkpointing","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"utils_train.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/utils_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3441023d420317ab"}},{"code_sha256_prefix":"c9031db20e2d11cc","entry":"make_progress_bar","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"utils_train.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/utils_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c9031db20e2d11cc"}},{"code_sha256_prefix":"a91dc4fa6fd3684d","entry":"safe_exp","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/utils_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a91dc4fa6fd3684d"}},{"code_sha256_prefix":"34cc47ee17f6c9ee","entry":"safe_log","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/utils_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34cc47ee17f6c9ee"}},{"code_sha256_prefix":"1b5fa89b375364a0","entry":"sandwich","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/utils_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1b5fa89b375364a0"}},{"code_sha256_prefix":"b6858c53f43787dc","entry":"zero_init","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_vdm.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_vdm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6858c53f43787dc"}},{"code_sha256_prefix":"fe4eb545bbb728e0","entry":"conv_nd","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_improved.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_improved.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe4eb545bbb728e0"}},{"code_sha256_prefix":"a863803cdd5f3ce6","entry":"make_master_params","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_improved.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_improved.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a863803cdd5f3ce6"}},{"code_sha256_prefix":"30e43bcf12d042b0","entry":"unflatten_master_params","repo":"nnaisense/bayesian-flow-networks","repo_kind":"official","path":"networks/unet_improved.py","file_url":"https://github.com/nnaisense/bayesian-flow-networks/blob/HEAD/networks/unet_improved.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"30e43bcf12d042b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}