{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/shapeshifter-networks-cross-layer-parameter","title":"Neural Parameter Allocation Search","arxiv_id":"2006.10598","date":"2020-06-18","proceeding":"ICLR 2022 4","authors":["Bryan A. Plummer","Nikoli Dryden","Julius Frost","Torsten Hoefler","Kate Saenko"],"abstract":"Training neural networks requires increasing amounts of memory. Parameter sharing can reduce memory and communication costs, but existing methods assume networks have many identical layers and utilize hand-crafted sharing strategies that fail to generalize. We introduce Neural Parameter Allocation Search (NPAS), a novel task where the goal is to train a neural network given an arbitrary, fixed parameter budget. NPAS covers both low-budget regimes, which produce compact networks, as well as a novel high-budget regime, where additional capacity can be added to boost performance without increasing inference FLOPs. To address NPAS, we introduce Shapeshifter Networks (SSNs), which automatically learn where and how to share parameters in a network to support any parameter budget without requiring any changes to the architecture or loss function. NPAS and SSNs provide a complete framework for addressing generalized parameter sharing, and can also be combined with prior work for additional performance gains. We demonstrate the effectiveness of our approach using nine network architectures across four diverse tasks, including ImageNet classification and transformers.","url_abs":"https://arxiv.org/abs/2006.10598v4","url_pdf":"https://arxiv.org/pdf/2006.10598v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"shapeshifter-networks-cross-layer-parameter","repo_url":"https://github.com/bryanplummer/ssn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"phrase-grounding","task_name":"Phrase Grounding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.10598","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.10598"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bryanplummer/ssn","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e74fd927ee1d4ef5","entry":"ParameterCombiner","repo":"bryanplummer/ssn","repo_kind":"official","path":"models/param_bank.py","file_url":"https://github.com/bryanplummer/ssn/blob/HEAD/models/param_bank.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e74fd927ee1d4ef5"}},{"code_sha256_prefix":"58bda0bd90b24dc5","entry":"SharedParameters","repo":"bryanplummer/ssn","repo_kind":"official","path":"models/param_bank.py","file_url":"https://github.com/bryanplummer/ssn/blob/HEAD/models/param_bank.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58bda0bd90b24dc5"}},{"code_sha256_prefix":"c80714ceae8c503e","entry":"ParameterBank","repo":"bryanplummer/ssn","repo_kind":"official","path":"models/param_bank.py","file_url":"https://github.com/bryanplummer/ssn/blob/HEAD/models/param_bank.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c80714ceae8c503e"}},{"code_sha256_prefix":"79df5f3e2329b5d2","entry":"ParameterUpsampler","repo":"bryanplummer/ssn","repo_kind":"official","path":"models/param_bank.py","file_url":"https://github.com/bryanplummer/ssn/blob/HEAD/models/param_bank.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79df5f3e2329b5d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}