{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mechanistic-mode-connectivity","title":"Mechanistic Mode Connectivity","arxiv_id":"2211.08422","date":"2022-11-15","proceeding":null,"authors":["Ekdeep Singh Lubana","Eric J. Bigelow","Robert P. Dick","David Krueger","Hidenori Tanaka"],"abstract":"We study neural network loss landscapes through the lens of mode connectivity, the observation that minimizers of neural networks retrieved via training on a dataset are connected via simple paths of low loss. Specifically, we ask the following question: are minimizers that rely on different mechanisms for making their predictions connected via simple paths of low loss? We provide a definition of mechanistic similarity as shared invariances to input transformations and demonstrate that lack of linear connectivity between two models implies they use dissimilar mechanisms for making their predictions. Relevant to practice, this result helps us demonstrate that naive fine-tuning on a downstream dataset can fail to alter a model's mechanisms, e.g., fine-tuning can fail to eliminate a model's reliance on spurious attributes. Our analysis also motivates a method for targeted alteration of a model's mechanisms, named connectivity-based fine-tuning (CBFT), which we analyze using several synthetic datasets for the task of reducing a model's reliance on spurious attributes.","url_abs":"https://arxiv.org/abs/2211.08422v3","url_pdf":"https://arxiv.org/pdf/2211.08422v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mechanistic-mode-connectivity","repo_url":"https://github.com/ekdeepslubana/mmc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.08422","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.08422"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/EkdeepSLubana/MMC","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ekdeepslubana/mmc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b7200ea011c4e0be","entry":"net_train","repo":"ekdeepslubana/mmc","repo_kind":"official","path":"trainer.py","file_url":"https://github.com/ekdeepslubana/mmc/blob/HEAD/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7200ea011c4e0be"}},{"code_sha256_prefix":"0ca2eb27c9c2692c","entry":"CBFT","repo":"ekdeepslubana/mmc","repo_kind":"official","path":"mech_fine_tuning.py","file_url":"https://github.com/ekdeepslubana/mmc/blob/HEAD/mech_fine_tuning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ca2eb27c9c2692c"}},{"code_sha256_prefix":"7f561d3453e03682","entry":"fine_tuning","repo":"ekdeepslubana/mmc","repo_kind":"official","path":"mech_fine_tuning.py","file_url":"https://github.com/ekdeepslubana/mmc/blob/HEAD/mech_fine_tuning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f561d3453e03682"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}