{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/batchensemble-an-alternative-approach-to-1","title":"BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning","arxiv_id":"2002.06715","date":"2020-02-17","proceeding":"ICLR 2020 1","authors":["Yeming Wen","Dustin Tran","Jimmy Ba"],"abstract":"Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and predictive uncertainty of single neural networks. However, an ensemble's cost for both training and testing increases linearly with the number of networks, which quickly becomes untenable. In this paper, we propose BatchEnsemble, an ensemble method whose computational and memory costs are significantly lower than typical ensembles. BatchEnsemble achieves this by defining each weight matrix to be the Hadamard product of a shared weight among all ensemble members and a rank-one matrix per member. Unlike ensembles, BatchEnsemble is not only parallelizable across devices, where one device trains one member, but also parallelizable within a device, where multiple ensemble members are updated simultaneously for a given mini-batch. Across CIFAR-10, CIFAR-100, WMT14 EN-DE/EN-FR translation, and out-of-distribution tasks, BatchEnsemble yields competitive accuracy and uncertainties as typical ensembles; the speedup at test time is 3X and memory reduction is 3X at an ensemble of size 4. We also apply BatchEnsemble to lifelong learning, where on Split-CIFAR-100, BatchEnsemble yields comparable performance to progressive neural networks while having a much lower computational and memory costs. We further show that BatchEnsemble can easily scale up to lifelong learning on Split-ImageNet which involves 100 sequential learning tasks.","url_abs":"https://arxiv.org/abs/2002.06715v2","url_pdf":"https://arxiv.org/pdf/2002.06715v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"batchensemble-an-alternative-approach-to-1","repo_url":"https://github.com/google/edward2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"batchensemble-an-alternative-approach-to-1","repo_url":"https://github.com/alexlimh/DPR_MUF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"batchensemble-an-alternative-approach-to-1","repo_url":"https://github.com/giannifranchi/LP_BNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"batchensemble-an-alternative-approach-to-1","repo_url":"https://github.com/nikitadurasov/masksembles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"batchensemble-an-alternative-approach-to-1","repo_url":"https://github.com/ensta-u2is/torch-uncertainty","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"lifelong-learning","task_name":"Lifelong learning"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"de-en","task_name":"de-en"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2002.06715","atlas_url":"https://app.syntology.ai/?focus=2002.06715","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.06715"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/edward2","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alexlimh/DPR_MUF","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ensta-u2is/torch-uncertainty","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nikitadurasov/masksembles","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/giannifranchi/LP_BNN","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"listed":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"2160fcc4e21a07c6","entry":"tile","repo":"giannifranchi/LP_BNN","repo_kind":"listed","path":"main_BatchEnsemble.py","file_url":"https://github.com/giannifranchi/LP_BNN/blob/HEAD/main_BatchEnsemble.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2160fcc4e21a07c6"}},{"code_sha256_prefix":"11da547728f79c87","entry":"getNetwork","repo":"giannifranchi/LP_BNN","repo_kind":"listed","path":"main_BatchEnsemble.py","file_url":"https://github.com/giannifranchi/LP_BNN/blob/HEAD/main_BatchEnsemble.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"11da547728f79c87"}},{"code_sha256_prefix":"9a41670af1d421ef","entry":"getNetwork","repo":"giannifranchi/LP_BNN","repo_kind":"listed","path":"evaluate_uncertainty.py","file_url":"https://github.com/giannifranchi/LP_BNN/blob/HEAD/evaluate_uncertainty.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a41670af1d421ef"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}