{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flipout-efficient-pseudo-independent-weight","title":"Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches","arxiv_id":"1803.04386","date":"2018-03-12","proceeding":"ICLR 2018 1","authors":["Yeming Wen","Paul Vicol","Jimmy Ba","Dustin Tran","Roger Grosse"],"abstract":"Stochastic neural net weights are used in a variety of contexts, including\nregularization, Bayesian neural nets, exploration in reinforcement learning,\nand evolution strategies. Unfortunately, due to the large number of weights,\nall the examples in a mini-batch typically share the same weight perturbation,\nthereby limiting the variance reduction effect of large mini-batches. We\nintroduce flipout, an efficient method for decorrelating the gradients within a\nmini-batch by implicitly sampling pseudo-independent weight perturbations for\neach example. Empirically, flipout achieves the ideal linear variance reduction\nfor fully connected networks, convolutional networks, and RNNs. We find\nsignificant speedups in training neural networks with multiplicative Gaussian\nperturbations. We show that flipout is effective at regularizing LSTMs, and\noutperforms previous methods. Flipout also enables us to vectorize evolution\nstrategies: in our experiments, a single GPU with flipout can handle the same\nthroughput as at least 40 CPU cores using existing methods, equivalent to a\nfactor-of-4 cost reduction on Amazon Web Services.","url_abs":"http://arxiv.org/abs/1803.04386v2","url_pdf":"http://arxiv.org/pdf/1803.04386v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flipout-efficient-pseudo-independent-weight","repo_url":"https://github.com/IntelLabs/bayesian-torch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"flipout-efficient-pseudo-independent-weight","repo_url":"https://github.com/dave-fernandes/ECGClassifier","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"flipout-efficient-pseudo-independent-weight","repo_url":"https://github.com/yamada-github-account/LearnBayesNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1803.04386","atlas_url":"https://app.syntology.ai/?focus=1803.04386","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.04386"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yamada-github-account/LearnBayesNN","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dave-fernandes/ECGClassifier","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IntelLabs/bayesian-torch","reach":null}],"summary":{"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"befe4ccdb8a4244e","entry":"validate","repo":"IntelLabs/bayesian-torch","repo_kind":"listed","path":"bayesian_torch/examples/main_bayesian_flipout_cifar.py","file_url":"https://github.com/IntelLabs/bayesian-torch/blob/HEAD/bayesian_torch/examples/main_bayesian_flipout_cifar.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"befe4ccdb8a4244e"}},{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}