{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/equivariant-deep-weight-space-alignment","title":"Equivariant Deep Weight Space Alignment","arxiv_id":"2310.13397","date":"2023-10-20","proceeding":null,"authors":["Aviv Navon","Aviv Shamsian","Ethan Fetaya","Gal Chechik","Nadav Dym","Haggai Maron"],"abstract":"Permutation symmetries of deep networks make basic operations like model merging and similarity estimation challenging. In many cases, aligning the weights of the networks, i.e., finding optimal permutations between their weights, is necessary. Unfortunately, weight alignment is an NP-hard problem. Prior research has mainly focused on solving relaxed versions of the alignment problem, leading to either time-consuming methods or sub-optimal solutions. To accelerate the alignment process and improve its quality, we propose a novel framework aimed at learning to solve the weight alignment problem, which we name Deep-Align. To that end, we first prove that weight alignment adheres to two fundamental symmetries and then, propose a deep architecture that respects these symmetries. Notably, our framework does not require any labeled data. We provide a theoretical analysis of our approach and evaluate Deep-Align on several types of network architectures and learning setups. Our experimental results indicate that a feed-forward pass with Deep-Align produces better or equivalent alignments compared to those produced by current optimization algorithms. Additionally, our alignments can be used as an effective initialization for other methods, leading to improved solutions with a significant speedup in convergence.","url_abs":"https://arxiv.org/abs/2310.13397v4","url_pdf":"https://arxiv.org/pdf/2310.13397v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"equivariant-deep-weight-space-alignment","repo_url":"https://github.com/AvivNavon/deep-align","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.13397","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.13397"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/AvivNavon/deep-align","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2,"ran_violates":1,"unverified":6},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fc6bfd01faa5f8ea","entry":"FCNet","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc6bfd01faa5f8ea"}},{"code_sha256_prefix":"834bac46e740376e","entry":"avg_weights_and_biases","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"834bac46e740376e"}},{"code_sha256_prefix":"ea8902cb68a8b695","entry":"permute_weights","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea8902cb68a8b695"}},{"code_sha256_prefix":"872a1d3182fa7818","entry":"unsqueeze_like","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"872a1d3182fa7818"}},{"code_sha256_prefix":"6c3e5634d197618e","entry":"CLFBatchSinkhorn","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c3e5634d197618e"}},{"code_sha256_prefix":"25782a75daea5300","entry":"Sinkhorn","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25782a75daea5300"}},{"code_sha256_prefix":"e2e87fef0fcfac5f","entry":"cnn_permutation_spec","repo":"avivnavon/deep-align","repo_kind":"official","path":"deepalign/matching.py","file_url":"https://github.com/avivnavon/deep-align/blob/HEAD/deepalign/matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2e87fef0fcfac5f"}},{"code_sha256_prefix":"c363c18932aa716b","entry":"matching","repo":"AvivNavon/deep-align","repo_kind":"official","path":"deepalign/sinkhorn.py","file_url":"https://github.com/AvivNavon/deep-align/blob/HEAD/deepalign/sinkhorn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c363c18932aa716b"}},{"code_sha256_prefix":"6e54909ba0908215","entry":"mlp_permutation_spec","repo":"avivnavon/deep-align","repo_kind":"official","path":"deepalign/matching.py","file_url":"https://github.com/avivnavon/deep-align/blob/HEAD/deepalign/matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e54909ba0908215"}},{"code_sha256_prefix":"2fdfe3a9f9b65da9","entry":"permutation_spec_from_axes_to_perm","repo":"avivnavon/deep-align","repo_kind":"official","path":"deepalign/matching.py","file_url":"https://github.com/avivnavon/deep-align/blob/HEAD/deepalign/matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fdfe3a9f9b65da9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}