{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/permutation-equivariant-neural-functionals-1","title":"Permutation Equivariant Neural Functionals","arxiv_id":"2302.14040","date":"2023-02-27","proceeding":"NeurIPS 2023 11","authors":["Allan Zhou","KaiEn Yang","Kaylee Burns","Adriano Cardace","Yiding Jiang","Samuel Sokota","J. Zico Kolter","Chelsea Finn"],"abstract":"This work studies the design of neural networks that can process the weights or gradients of other neural networks, which we refer to as neural functional networks (NFNs). Despite a wide range of potential applications, including learned optimization, processing implicit neural representations, network editing, and policy evaluation, there are few unifying principles for designing effective architectures that process the weights of other networks. We approach the design of neural functionals through the lens of symmetry, in particular by focusing on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. We introduce a framework for building permutation equivariant neural functionals, whose architectures encode these symmetries as an inductive bias. The key building blocks of this framework are NF-Layers (neural functional layers) that we constrain to be permutation equivariant through an appropriate parameter sharing scheme. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks that require processing the weights of MLPs and CNNs, such as predicting classifier generalization, producing \"winning ticket\" sparsity masks for initializations, and classifying or editing implicit neural representations (INRs). In addition, we provide code for our models and experiments at https://github.com/AllanYangZhou/nfn.","url_abs":"https://arxiv.org/abs/2302.14040v3","url_pdf":"https://arxiv.org/pdf/2302.14040v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"permutation-equivariant-neural-functionals-1","repo_url":"https://github.com/allanyangzhou/nfn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"permutation-equivariant-neural-functionals-1","repo_url":"https://github.com/jkalogero/scalegmn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2302.14040","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.14040"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jkalogero/scalegmn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allanyangzhou/nfn","reach":null}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"25b277b54e5787ff","entry":"InvariantLayer","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25b277b54e5787ff"}},{"code_sha256_prefix":"59052528a73a219f","entry":"MLPNet","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59052528a73a219f"}},{"code_sha256_prefix":"355080531ee83dab","entry":"NetworkSpec","repo":"allanyangzhou/nfn","repo_kind":"official","path":"nfn/layers/layers.py","file_url":"https://github.com/allanyangzhou/nfn/blob/HEAD/nfn/layers/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"355080531ee83dab"}},{"code_sha256_prefix":"a0e7d080ac1bf58c","entry":"Sine","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0e7d080ac1bf58c"}},{"code_sha256_prefix":"313b13b7bd56f7cf","entry":"choose_activation","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"313b13b7bd56f7cf"}},{"code_sha256_prefix":"67d3763fe4254040","entry":"mlp","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67d3763fe4254040"}},{"code_sha256_prefix":"da9f6cc5270f3205","entry":"EquivariantLayer","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da9f6cc5270f3205"}},{"code_sha256_prefix":"48fdd1872aded0c3","entry":"EquivariantNet","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48fdd1872aded0c3"}},{"code_sha256_prefix":"0d5da287e13d8e84","entry":"InvariantNet","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/layers.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d5da287e13d8e84"}},{"code_sha256_prefix":"af80d775104f9282","entry":"NPLinear","repo":"allanyangzhou/nfn","repo_kind":"official","path":"nfn/layers/layers.py","file_url":"https://github.com/allanyangzhou/nfn/blob/HEAD/nfn/layers/layers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af80d775104f9282"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}