{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deterministic-variational-inference-for","title":"Deterministic Variational Inference for Robust Bayesian Neural Networks","arxiv_id":"1810.03958","date":"2018-10-09","proceeding":"ICLR 2019 5","authors":["Anqi Wu","Sebastian Nowozin","Edward Meeds","Richard E. Turner","José Miguel Hernández-Lobato","Alexander L. Gaunt"],"abstract":"Bayesian neural networks (BNNs) hold great promise as a flexible and\nprincipled solution to deal with uncertainty when learning from finite data.\nAmong approaches to realize probabilistic inference in deep neural networks,\nvariational Bayes (VB) is theoretically grounded, generally applicable, and\ncomputationally efficient. With wide recognition of potential advantages, why\nis it that variational Bayes has seen very limited practical use for BNNs in\nreal applications? We argue that variational inference in neural networks is\nfragile: successful implementations require careful initialization and tuning\nof prior variances, as well as controlling the variance of Monte Carlo gradient\nestimates. We provide two innovations that aim to turn VB into a robust\ninference tool for Bayesian neural networks: first, we introduce a novel\ndeterministic method to approximate moments in neural networks, eliminating\ngradient variance; second, we introduce a hierarchical prior for parameters and\na novel Empirical Bayes procedure for automatically selecting prior variances.\nCombining these two innovations, the resulting method is highly efficient and\nrobust. On the application of heteroscedastic regression we demonstrate good\npredictive performance over alternative approaches.","url_abs":"http://arxiv.org/abs/1810.03958v2","url_pdf":"http://arxiv.org/pdf/1810.03958v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deterministic-variational-inference-for","repo_url":"https://github.com/Microsoft/deterministic-variational-inference","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deterministic-variational-inference-for","repo_url":"https://github.com/markovalexander/DVI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deterministic-variational-inference-for","repo_url":"https://github.com/omwright/cov-prop-nn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.03958","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1810.03958"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Microsoft/deterministic-variational-inference","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/omwright/cov-prop-nn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/markovalexander/DVI","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":23},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1},"listed":{"samples":16,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e53a59f1614c0037","entry":"autoload_csv","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e53a59f1614c0037"}},{"code_sha256_prefix":"4f0d31ec0d21de1c","entry":"gaussian_cdf","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"bayes_util.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/bayes_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f0d31ec0d21de1c"}},{"code_sha256_prefix":"cdfb9235c506423b","entry":"gaussian_cdf","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/bayesian_utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/bayesian_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdfb9235c506423b"}},{"code_sha256_prefix":"e8e6d80b1447469d","entry":"gaussian_cdf","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_tightness_mnist.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_tightness_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8e6d80b1447469d"}},{"code_sha256_prefix":"9d250aca9cbb3e9d","entry":"generate_uci_data","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d250aca9cbb3e9d"}},{"code_sha256_prefix":"35c6488e12624aac","entry":"get_device_string","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35c6488e12624aac"}},{"code_sha256_prefix":"1c9fe43e56ed658f","entry":"get_hypers","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c9fe43e56ed658f"}},{"code_sha256_prefix":"b31269ddf9e7a14b","entry":"get_mse","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_dvi.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_dvi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b31269ddf9e7a14b"}},{"code_sha256_prefix":"3628b68aed79c0e2","entry":"get_predictions","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3628b68aed79c0e2"}},{"code_sha256_prefix":"a9439e641c4c1ff4","entry":"glorot_init","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"gaussian_variables.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/gaussian_variables.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9439e641c4c1ff4"}},{"code_sha256_prefix":"5399bc831fe81e9b","entry":"matrix_diag_part","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/bayesian_utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/bayesian_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5399bc831fe81e9b"}},{"code_sha256_prefix":"1aa2aea36b573636","entry":"one_hot_encoding","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1aa2aea36b573636"}},{"code_sha256_prefix":"22db9db360759a69","entry":"pred2acc","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22db9db360759a69"}},{"code_sha256_prefix":"8cc2f994e1088d1e","entry":"restrict_dataset_size","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8cc2f994e1088d1e"}},{"code_sha256_prefix":"253775c48c31a4e3","entry":"softrelu","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"bayes_util.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/bayes_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"253775c48c31a4e3"}},{"code_sha256_prefix":"3a80ea713ce9c133","entry":"softrelu","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_tightness_mnist.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_tightness_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a80ea713ce9c133"}},{"code_sha256_prefix":"3d5e81c357b27586","entry":"softrelu","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"dvi/bayes_utils.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/dvi/bayes_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d5e81c357b27586"}},{"code_sha256_prefix":"4b42f948c28568f5","entry":"standard_gaussian","repo":"Microsoft/deterministic-variational-inference","repo_kind":"official","path":"bayes_util.py","file_url":"https://github.com/Microsoft/deterministic-variational-inference/blob/HEAD/bayes_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b42f948c28568f5"}},{"code_sha256_prefix":"2f1e2e89b495caf7","entry":"standard_gaussian","repo":"markovalexander/DVI","repo_kind":"listed","path":"core/bayesian_utils.py","file_url":"https://github.com/markovalexander/DVI/blob/HEAD/core/bayesian_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f1e2e89b495caf7"}},{"code_sha256_prefix":"229ae3d179dcefa5","entry":"standard_gaussian","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_tightness_mnist.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_tightness_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"229ae3d179dcefa5"}},{"code_sha256_prefix":"57b45b3540c780fe","entry":"test","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"train_mnist.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/train_mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57b45b3540c780fe"}},{"code_sha256_prefix":"d1a76c1ba3b77dec","entry":"train_epoch","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_dvi.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_dvi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1a76c1ba3b77dec"}},{"code_sha256_prefix":"fafc757982dcd649","entry":"validate_model","repo":"omwright/cov-prop-nn","repo_kind":"listed","path":"run_dvi.py","file_url":"https://github.com/omwright/cov-prop-nn/blob/HEAD/run_dvi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fafc757982dcd649"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}