{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/wide-mean-field-bayesian-neural-networks","title":"Wide Mean-Field Bayesian Neural Networks Ignore the Data","arxiv_id":"2202.11670","date":"2022-02-23","proceeding":null,"authors":["Beau Coker","Wessel P. Bruinsma","David R. Burt","Weiwei Pan","Finale Doshi-Velez"],"abstract":"Bayesian neural networks (BNNs) combine the expressive power of deep learning with the advantages of Bayesian formalism. In recent years, the analysis of wide, deep BNNs has provided theoretical insight into their priors and posteriors. However, we have no analogous insight into their posteriors under approximate inference. In this work, we show that mean-field variational inference entirely fails to model the data when the network width is large and the activation function is odd. Specifically, for fully-connected BNNs with odd activation functions and a homoscedastic Gaussian likelihood, we show that the optimal mean-field variational posterior predictive (i.e., function space) distribution converges to the prior predictive distribution as the width tends to infinity. We generalize aspects of this result to other likelihoods. Our theoretical results are suggestive of underfitting behavior previously observered in BNNs. While our convergence bounds are non-asymptotic and constants in our analysis can be computed, they are currently too loose to be applicable in standard training regimes. Finally, we show that the optimal approximate posterior need not tend to the prior if the activation function is not odd, showing that our statements cannot be generalized arbitrarily.","url_abs":"https://arxiv.org/abs/2202.11670v1","url_pdf":"https://arxiv.org/pdf/2202.11670v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"wide-mean-field-bayesian-neural-networks","repo_url":"https://github.com/dtak/wide-bnns-public","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.11670","atlas_url":"https://app.syntology.ai/?focus=2202.11670","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.11670"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dtak/wide-bnns-public","reach":null}],"summary":{"ran_draft_wrong":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a0a0864fe2c66299","entry":"plot_predictive","repo":"dtak/wide-bnns-public","repo_kind":"official","path":"experiments/figures/work/make_figures.py","file_url":"https://github.com/dtak/wide-bnns-public/blob/HEAD/experiments/figures/work/make_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0a0864fe2c66299"}},{"code_sha256_prefix":"6a2fcddfac274eb4","entry":"plot_upcrossings","repo":"dtak/wide-bnns-public","repo_kind":"official","path":"experiments/figures/work/make_figures.py","file_url":"https://github.com/dtak/wide-bnns-public/blob/HEAD/experiments/figures/work/make_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a2fcddfac274eb4"}},{"code_sha256_prefix":"e9c63691c59a1449","entry":"scalable_foward","repo":"dtak/wide-bnns-public","repo_kind":"official","path":"scripts/experiment_pytorch.py","file_url":"https://github.com/dtak/wide-bnns-public/blob/HEAD/scripts/experiment_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9c63691c59a1449"}},{"code_sha256_prefix":"697d6e902861b9c5","entry":"compute_metrics","repo":"dtak/wide-bnns-public","repo_kind":"official","path":"scripts/experiment_pytorch.py","file_url":"https://github.com/dtak/wide-bnns-public/blob/HEAD/scripts/experiment_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"697d6e902861b9c5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}