{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-independent-subnetworks-for-robust-1","title":"Training independent subnetworks for robust prediction","arxiv_id":"2010.06610","date":"2020-10-13","proceeding":"ICLR 2021 1","authors":["Marton Havasi","Rodolphe Jenatton","Stanislav Fort","Jeremiah Zhe Liu","Jasper Snoek","Balaji Lakshminarayanan","Andrew M. Dai","Dustin Tran"],"abstract":"Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network. However, these methods still require multiple forward passes for prediction, leading to a significant computational cost. In this work, we show a surprising result: the benefits of using multiple predictions can be achieved `for free' under a single model's forward pass. In particular, we show that, using a multi-input multi-output (MIMO) configuration, one can utilize a single model's capacity to train multiple subnetworks that independently learn the task at hand. By ensembling the predictions made by the subnetworks, we improve model robustness without increasing compute. We observe a significant improvement in negative log-likelihood, accuracy, and calibration error on CIFAR10, CIFAR100, ImageNet, and their out-of-distribution variants compared to previous methods.","url_abs":"https://arxiv.org/abs/2010.06610v2","url_pdf":"https://arxiv.org/pdf/2010.06610v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-independent-subnetworks-for-robust-1","repo_url":"https://github.com/google/uncertainty-baselines","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"training-independent-subnetworks-for-robust-1","repo_url":"https://github.com/ensta-u2is/torch-uncertainty","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2010.06610","atlas_url":"https://app.syntology.ai/?focus=2010.06610","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.06610"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/uncertainty-baselines","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ensta-u2is/torch-uncertainty","reach":null}],"summary":{"ran":1,"ran_honours":3,"unverified":1},"by_repo_kind":{"listed":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8d8d122cf73afa3e","entry":"Mask1d","repo":"ensta-u2is/torch-uncertainty","repo_kind":"listed","path":"src/torch_uncertainty/layers/masksembles.py","file_url":"https://github.com/ensta-u2is/torch-uncertainty/blob/HEAD/src/torch_uncertainty/layers/masksembles.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8d8d122cf73afa3e"}},{"code_sha256_prefix":"6d8a981a433b52d5","entry":"_generate_masks","repo":"ensta-u2is/torch-uncertainty","repo_kind":"listed","path":"src/torch_uncertainty/layers/masksembles.py","file_url":"https://github.com/ensta-u2is/torch-uncertainty/blob/HEAD/src/torch_uncertainty/layers/masksembles.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6d8a981a433b52d5"}},{"code_sha256_prefix":"5ca5c0443875a99b","entry":"generate_masks","repo":"ensta-u2is/torch-uncertainty","repo_kind":"listed","path":"src/torch_uncertainty/layers/masksembles.py","file_url":"https://github.com/ensta-u2is/torch-uncertainty/blob/HEAD/src/torch_uncertainty/layers/masksembles.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ca5c0443875a99b"}},{"code_sha256_prefix":"11e6d40e826ec772","entry":"generation_wrapper","repo":"ensta-u2is/torch-uncertainty","repo_kind":"listed","path":"src/torch_uncertainty/layers/masksembles.py","file_url":"https://github.com/ensta-u2is/torch-uncertainty/blob/HEAD/src/torch_uncertainty/layers/masksembles.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"11e6d40e826ec772"}},{"code_sha256_prefix":"10cca0803fd92522","entry":"MaskedLinear","repo":"ensta-u2is/torch-uncertainty","repo_kind":"listed","path":"src/torch_uncertainty/layers/masksembles.py","file_url":"https://github.com/ensta-u2is/torch-uncertainty/blob/HEAD/src/torch_uncertainty/layers/masksembles.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"10cca0803fd92522"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}