{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/equivariant-architectures-for-learning-in","title":"Equivariant Architectures for Learning in Deep Weight Spaces","arxiv_id":"2301.12780","date":"2023-01-30","proceeding":null,"authors":["Aviv Navon","Aviv Shamsian","Idan Achituve","Ethan Fetaya","Gal Chechik","Haggai Maron"],"abstract":"Designing machine learning architectures for processing neural networks in their raw weight matrix form is a newly introduced research direction. Unfortunately, the unique symmetry structure of deep weight spaces makes this design very challenging. If successful, such architectures would be capable of performing a wide range of intriguing tasks, from adapting a pre-trained network to a new domain to editing objects represented as functions (INRs or NeRFs). As a first step towards this goal, we present here a novel network architecture for learning in deep weight spaces. It takes as input a concatenation of weights and biases of a pre-trained MLP and processes it using a composition of layers that are equivariant to the natural permutation symmetry of the MLP's weights: Changing the order of neurons in intermediate layers of the MLP does not affect the function it represents. We provide a full characterization of all affine equivariant and invariant layers for these symmetries and show how these layers can be implemented using three basic operations: pooling, broadcasting, and fully connected layers applied to the input in an appropriate manner. We demonstrate the effectiveness of our architecture and its advantages over natural baselines in a variety of learning tasks.","url_abs":"https://arxiv.org/abs/2301.12780v2","url_pdf":"https://arxiv.org/pdf/2301.12780v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"equivariant-architectures-for-learning-in","repo_url":"https://github.com/AvivNavon/DWSNets","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"equivariant-architectures-for-learning-in","repo_url":"https://github.com/jkalogero/scalegmn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2301.12780","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.12780"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jkalogero/scalegmn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AvivNavon/DWSNets","reach":null}],"summary":{"ran":14,"ran_draft_wrong":2,"ran_fixture":1,"unverified":17},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1},"listed":{"samples":19,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b7b3c9250967458c","entry":"Attn","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7b3c9250967458c"}},{"code_sha256_prefix":"749cec24623f3134","entry":"BaseLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"749cec24623f3134"}},{"code_sha256_prefix":"fbbb08156decace0","entry":"DeepSet","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbbb08156decace0"}},{"code_sha256_prefix":"c5e8282ae61f6295","entry":"FromFirstLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5e8282ae61f6295"}},{"code_sha256_prefix":"0dcb212b6ae589d6","entry":"FromLastLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0dcb212b6ae589d6"}},{"code_sha256_prefix":"99b2d300b891ce85","entry":"GraphInit","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99b2d300b891ce85"}},{"code_sha256_prefix":"dfdb026b20ec9c69","entry":"MAB","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfdb026b20ec9c69"}},{"code_sha256_prefix":"69443da94832c609","entry":"NonNeighborInternalLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69443da94832c609"}},{"code_sha256_prefix":"7929083636f127c7","entry":"SAB","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7929083636f127c7"}},{"code_sha256_prefix":"258566dbabd1a7e5","entry":"SetKroneckerSetLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"258566dbabd1a7e5"}},{"code_sha256_prefix":"5bdb738dcf56334c","entry":"SetLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bdb738dcf56334c"}},{"code_sha256_prefix":"621ace76d6201550","entry":"SineUpdate","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"621ace76d6201550"}},{"code_sha256_prefix":"24e755539712d3aa","entry":"ToFirstLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24e755539712d3aa"}},{"code_sha256_prefix":"fe81153f8418e6f3","entry":"ToLastLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe81153f8418e6f3"}},{"code_sha256_prefix":"adc3a85b7d5652a3","entry":"get_edge_types","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"adc3a85b7d5652a3"}},{"code_sha256_prefix":"7d466a0a2fae42ba","entry":"get_node_types","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d466a0a2fae42ba"}},{"code_sha256_prefix":"f8f76184e7d43e05","entry":"graph_to_wb","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8f76184e7d43e05"}},{"code_sha256_prefix":"2a78602f4718460d","entry":"BaseScaleGMN","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a78602f4718460d"}},{"code_sha256_prefix":"9c77538d64ffc753","entry":"EdgeUpdate","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c77538d64ffc753"}},{"code_sha256_prefix":"b1bbaf40374412c7","entry":"GNN_layer","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1bbaf40374412c7"}},{"code_sha256_prefix":"854d0920747d8ccc","entry":"GNN_layer_aggr","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"854d0920747d8ccc"}},{"code_sha256_prefix":"18f6ecf173b8d90f","entry":"GeneralMatrixSetLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18f6ecf173b8d90f"}},{"code_sha256_prefix":"a1787eb42cb85a8a","entry":"GeneralSetLayer","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1787eb42cb85a8a"}},{"code_sha256_prefix":"949b907b6475fde9","entry":"PermScaleInvariantReadout","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"949b907b6475fde9"}},{"code_sha256_prefix":"6d0190116d5a66fb","entry":"PositionalEncoding","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d0190116d5a66fb"}},{"code_sha256_prefix":"763b5c0f24526e0c","entry":"ScaleEq_GNN_layer","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"763b5c0f24526e0c"}},{"code_sha256_prefix":"ce35a061c9b59121","entry":"ScaleGMN_GNN","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce35a061c9b59121"}},{"code_sha256_prefix":"c73057a5d50237ba","entry":"ScaleGMN_GNN_bidir","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c73057a5d50237ba"}},{"code_sha256_prefix":"fa9b3269eb251c79","entry":"ScaleGMN_GNN_fw","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fa9b3269eb251c79"}},{"code_sha256_prefix":"4bc57d760e6c8aa1","entry":"ScaleGMN_GNN_layer_aggr","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4bc57d760e6c8aa1"}},{"code_sha256_prefix":"b9118d28b09d5b70","entry":"ScaleGMN_equiv","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9118d28b09d5b70"}},{"code_sha256_prefix":"ec1054fbb47b1286","entry":"WeightToWeightBlock","repo":"AvivNavon/DWSNets","repo_kind":"official","path":"nn/layers/weight_to_weight.py","file_url":"https://github.com/AvivNavon/DWSNets/blob/HEAD/nn/layers/weight_to_weight.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec1054fbb47b1286"}},{"code_sha256_prefix":"af6555f8eb5878c6","entry":"base_GNN_layer","repo":"jkalogero/scalegmn","repo_kind":"listed","path":"src/scalegmn/models.py","file_url":"https://github.com/jkalogero/scalegmn/blob/HEAD/src/scalegmn/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af6555f8eb5878c6"}},{"code_sha256_prefix":"4d48f663c05399f4","entry":"evaluate","repo":"avivnavon/dwsnets","repo_kind":"official","path":"experiments/mnist/trainer.py","file_url":"https://github.com/avivnavon/dwsnets/blob/HEAD/experiments/mnist/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d48f663c05399f4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}