{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variational-dropout-and-the-local","title":"Variational Dropout and the Local Reparameterization Trick","arxiv_id":"1506.02557","date":"2015-06-08","proceeding":"NeurIPS 2015 12","authors":["Diederik P. Kingma","Tim Salimans","Max Welling"],"abstract":"We investigate a local reparameterizaton technique for greatly reducing the\nvariance of stochastic gradients for variational Bayesian inference (SGVB) of a\nposterior over model parameters, while retaining parallelizability. This local\nreparameterization translates uncertainty about global parameters into local\nnoise that is independent across datapoints in the minibatch. Such\nparameterizations can be trivially parallelized and have variance that is\ninversely proportional to the minibatch size, generally leading to much faster\nconvergence. Additionally, we explore a connection with dropout: Gaussian\ndropout objectives correspond to SGVB with local reparameterization, a\nscale-invariant prior and proportionally fixed posterior variance. Our method\nallows inference of more flexibly parameterized posteriors; specifically, we\npropose variational dropout, a generalization of Gaussian dropout where the\ndropout rates are learned, often leading to better models. The method is\ndemonstrated through several experiments.","url_abs":"http://arxiv.org/abs/1506.02557v2","url_pdf":"http://arxiv.org/pdf/1506.02557v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/Anou9531/Bayesian-CNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/COMP6248-Reproducability-Challenge/Reproduction-of-Probabilistic-binary-neural-networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/ThirstyScholar/bayes-by-backprop","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/dennysemko/VectorizedBayesByBackprop","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/gpapamak/bayesian_neural_networks_demo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/jamesvuc/BBVI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/kumar-shridhar/BayesianConvNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/kumar-shridhar/PyTorch-BayesianCNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/liqichen6688/baycnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/nomercy77/Implementing-Bayesian-CNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/tennisonliu/bayesian-neural-network","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"variational-dropout-and-the-local","repo_url":"https://github.com/tyxe-bdl/tyxe","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1506.02557","atlas_url":"https://app.syntology.ai/?focus=1506.02557","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1506.02557"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/COMP6248-Reproducability-Challenge/Reproduction-of-Probabilistic-binary-neural-networks","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dennysemko/VectorizedBayesByBackprop","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liqichen6688/baycnn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kumar-shridhar/PyTorch-BayesianCNN","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kumar-shridhar/BayesianConvNet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ThirstyScholar/bayes-by-backprop","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Anou9531/Bayesian-CNN","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jamesvuc/BBVI","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tyxe-bdl/tyxe","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nomercy77/Implementing-Bayesian-CNN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tennisonliu/bayesian-neural-network","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gpapamak/bayesian_neural_networks_demo","reach":null}],"summary":{"ran_draft_wrong":4,"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":4,"ran":3,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"f6fa0a64369c93e3","entry":"calc_ece","repo":"tyxe-bdl/tyxe","repo_kind":"listed","path":"examples/gnn.py","file_url":"https://github.com/tyxe-bdl/tyxe/blob/HEAD/examples/gnn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6fa0a64369c93e3"}},{"code_sha256_prefix":"87cad5dc1e47c772","entry":"create_data_reg","repo":"tennisonliu/bayesian-neural-network","repo_kind":"listed","path":"utils/data_utils.py","file_url":"https://github.com/tennisonliu/bayesian-neural-network/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87cad5dc1e47c772"}},{"code_sha256_prefix":"248eed0714c8eb72","entry":"create_grid","repo":"gpapamak/bayesian_neural_networks_demo","repo_kind":"listed","path":"bnn_demo.py","file_url":"https://github.com/gpapamak/bayesian_neural_networks_demo/blob/HEAD/bnn_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"248eed0714c8eb72"}},{"code_sha256_prefix":"dd46eb517cc22519","entry":"get_uncertainty_per_batch","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"dd46eb517cc22519"}},{"code_sha256_prefix":"3698e0abeea9d5d0","entry":"get_uncertainty_per_image","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"3698e0abeea9d5d0"}},{"code_sha256_prefix":"b003df97da8b9b8f","entry":"read_data_rl","repo":"tennisonliu/bayesian-neural-network","repo_kind":"listed","path":"utils/data_utils.py","file_url":"https://github.com/tennisonliu/bayesian-neural-network/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b003df97da8b9b8f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}