{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/composable-interventions-for-language-models","title":"Composable Interventions for Language Models","arxiv_id":"2407.06483","date":"2024-07-09","proceeding":null,"authors":["Arinbjorn Kolbeinsson","Kyle O'Brien","Tianjin Huang","ShangHua Gao","Shiwei Liu","Jonathan Richard Schwarz","Anurag Vaidya","Faisal Mahmood","Marinka Zitnik","Tianlong Chen","Thomas Hartvigsen"],"abstract":"Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining. But despite a flood of new methods, different types of interventions are largely developing independently. In practice, multiple interventions must be applied sequentially to the same model, yet we lack standardized ways to study how interventions interact. We fill this gap by introducing composable interventions, a framework to study the effects of using multiple interventions on the same language models, featuring new metrics and a unified codebase. Using our framework, we conduct extensive experiments and compose popular methods from three emerging intervention categories -- Knowledge Editing, Model Compression, and Machine Unlearning. Our results from 310 different compositions uncover meaningful interactions: compression hinders editing and unlearning, composing interventions hinges on their order of application, and popular general-purpose metrics are inadequate for assessing composability. Taken together, our findings showcase clear gaps in composability, suggesting a need for new multi-objective interventions. All of our code is public: https://github.com/hartvigsen-group/composable-interventions.","url_abs":"https://arxiv.org/abs/2407.06483v1","url_pdf":"https://arxiv.org/pdf/2407.06483v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"composable-interventions-for-language-models","repo_url":"https://github.com/hartvigsen-group/composable-interventions","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"machine-unlearning","task_name":"Machine Unlearning"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"knowledge-editing","task_name":"knowledge editing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.06483","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.06483"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hartvigsen-group/composable-interventions","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"ran":4,"ran_fixture":1,"unverified":10},"by_repo_kind":{"official":{"samples":17,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"27659a9c234ffb11","entry":"binary_log_probs","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27659a9c234ffb11"}},{"code_sha256_prefix":"ae77bcc993ab559a","entry":"generate_fingerprint","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"run_edit.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/run_edit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae77bcc993ab559a"}},{"code_sha256_prefix":"dee8be97b2cf4692","entry":"get_wikitext2","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/data.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/data.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dee8be97b2cf4692"}},{"code_sha256_prefix":"920b394e7ad80c53","entry":"hierarchical_subsequence","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/util/nethook.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"920b394e7ad80c53"}},{"code_sha256_prefix":"e078eeed20f03838","entry":"masked_mean","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e078eeed20f03838"}},{"code_sha256_prefix":"70f6ab8bde55420e","entry":"recursive_copy","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/util/nethook.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70f6ab8bde55420e"}},{"code_sha256_prefix":"440ff98c2b1ae1aa","entry":"subsequence","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/util/nethook.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/util/nethook.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"440ff98c2b1ae1aa"}},{"code_sha256_prefix":"48bcc4bf8834c89a","entry":"get_args","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"main_quantize.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/main_quantize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48bcc4bf8834c89a"}},{"code_sha256_prefix":"823f33725cd8f75d","entry":"get_c4","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/data.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"823f33725cd8f75d"}},{"code_sha256_prefix":"1ebe223f71f863bc","entry":"get_c4","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/datautils.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ebe223f71f863bc"}},{"code_sha256_prefix":"e9098552690ff1e9","entry":"get_ga_data","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"ga_utils.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/ga_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e9098552690ff1e9"}},{"code_sha256_prefix":"d280312c1c709fbb","entry":"get_loaders","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/data.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d280312c1c709fbb"}},{"code_sha256_prefix":"d2e4c1979dd70b9a","entry":"get_model","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/trainer/models.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/trainer/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2e4c1979dd70b9a"}},{"code_sha256_prefix":"3a6ac4325dad94a4","entry":"get_ptb","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/datautils.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a6ac4325dad94a4"}},{"code_sha256_prefix":"2defe42cf1a7f301","entry":"get_tokenizer","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/trainer/models.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/trainer/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2defe42cf1a7f301"}},{"code_sha256_prefix":"ce2dbbe16d039626","entry":"get_wikitext2","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"sparsellm/lib/datautils.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/sparsellm/lib/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce2dbbe16d039626"}},{"code_sha256_prefix":"bcc592aa5c1e2bb3","entry":"kl_loc_loss","repo":"hartvigsen-group/composable-interventions","repo_kind":"official","path":"easyeditor/trainer/losses.py","file_url":"https://github.com/hartvigsen-group/composable-interventions/blob/HEAD/easyeditor/trainer/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bcc592aa5c1e2bb3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}