{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/counterfactual-generation-from-language","title":"Gumbel Counterfactual Generation From Language Models","arxiv_id":"2411.07180","date":"2024-11-11","proceeding":null,"authors":["Shauli Ravfogel","Anej Svete","Vésteinn Snæbjarnarson","Ryan Cotterell"],"abstract":"Understanding and manipulating the causal generation mechanisms in language models is essential for controlling their behavior. Previous work has primarily relied on techniques such as representation surgery -- e.g., model ablations or manipulation of linear subspaces tied to specific concepts -- to \\emph{intervene} on these models. To understand the impact of interventions precisely, it is useful to examine counterfactuals -- e.g., how a given sentence would have appeared had it been generated by the model following a specific intervention. We highlight that counterfactual reasoning is conceptually distinct from interventions, as articulated in Pearl's causal hierarchy. Based on this observation, we propose a framework for generating true string counterfactuals by reformulating language models as a structural equation model using the Gumbel-max trick, which we called Gumbel counterfactual generation. This reformulation allows us to model the joint distribution over original strings and their counterfactuals resulting from the same instantiation of the sampling noise. We develop an algorithm based on hindsight Gumbel sampling that allows us to infer the latent noise variables and generate counterfactuals of observed strings. Our experiments demonstrate that the approach produces meaningful counterfactuals while at the same time showing that commonly used intervention techniques have considerable undesired side effects.","url_abs":"https://arxiv.org/abs/2411.07180v3","url_pdf":"https://arxiv.org/pdf/2411.07180v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"counterfactual-generation-from-language","repo_url":"https://github.com/shauli-ravfogel/lm-counterfactuals","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"counterfactual-reasoning","task_name":"Counterfactual Reasoning"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[{"method_slug":"counterfactuals","method_name":"Counterfactuals"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.07180","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.07180"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/shauli-ravfogel/lm-counterfactuals","reach":null}],"summary":{"ran_honours":1,"ran_fixture":1,"ran_violates":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"b2683abe11b554e0","entry":"counterfactual_generation_vectorized","repo":"shauli-ravfogel/lm-counterfactuals","repo_kind":"official","path":"sampling.py","file_url":"https://github.com/shauli-ravfogel/lm-counterfactuals/blob/HEAD/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2683abe11b554e0"}},{"code_sha256_prefix":"19d0b12fed62b5cf","entry":"topdown_vectorized","repo":"shauli-ravfogel/lm-counterfactuals","repo_kind":"official","path":"sampling.py","file_url":"https://github.com/shauli-ravfogel/lm-counterfactuals/blob/HEAD/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19d0b12fed62b5cf"}},{"code_sha256_prefix":"b1748facc3177870","entry":"truncated_gumbel_vectorized","repo":"shauli-ravfogel/lm-counterfactuals","repo_kind":"official","path":"sampling.py","file_url":"https://github.com/shauli-ravfogel/lm-counterfactuals/blob/HEAD/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1748facc3177870"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}