{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uncovering-causal-variables-in-transformers","title":"Uncovering Intermediate Variables in Transformers using Circuit Probing","arxiv_id":"2311.04354","date":"2023-11-07","proceeding":null,"authors":["Michael A. Lepori","Thomas Serre","Ellie Pavlick"],"abstract":"Neural network models have achieved high performance on a wide variety of complex tasks, but the algorithms that they implement are notoriously difficult to interpret. In order to understand these algorithms, it is often necessary to hypothesize intermediate variables involved in the network's computation. For example, does a language model depend on particular syntactic properties when generating a sentence? However, existing analysis tools make it difficult to test hypotheses of this type. We propose a new analysis technique -- circuit probing -- that automatically uncovers low-level circuits that compute hypothesized intermediate variables. This enables causal analysis through targeted ablation at the level of model parameters. We apply this method to models trained on simple arithmetic tasks, demonstrating its effectiveness at (1) deciphering the algorithms that models have learned, (2) revealing modular structure within a model, and (3) tracking the development of circuits over training. We compare circuit probing to other methods across these three experiments, and find it on par or more effective than existing analysis methods. Finally, we demonstrate circuit probing on a real-world use case, uncovering circuits that are responsible for subject-verb agreement and reflexive anaphora in GPT2-Small and Medium.","url_abs":"https://arxiv.org/abs/2311.04354v2","url_pdf":"https://arxiv.org/pdf/2311.04354v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uncovering-causal-variables-in-transformers","repo_url":"https://github.com/mlepori1/circuit_probing","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.04354","atlas_url":"https://app.syntology.ai/?focus=2311.04354","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.04354"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mlepori1/circuit_probing","reach":null}],"summary":{"ran_honours":3,"ran_fixture":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"76eabbb99306f788","entry":"compute_acc","repo":"mlepori1/circuit_probing","repo_kind":"official","path":"src/train_algorithmic_models.py","file_url":"https://github.com/mlepori1/circuit_probing/blob/HEAD/src/train_algorithmic_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76eabbb99306f788"}},{"code_sha256_prefix":"abece20a3358d88f","entry":"eval_probe","repo":"mlepori1/circuit_probing","repo_kind":"official","path":"src/circuit_probing.py","file_url":"https://github.com/mlepori1/circuit_probing/blob/HEAD/src/circuit_probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"abece20a3358d88f"}},{"code_sha256_prefix":"55844d9233f79937","entry":"get_dataset_length","repo":"mlepori1/circuit_probing","repo_kind":"official","path":"src/circuit_probing.py","file_url":"https://github.com/mlepori1/circuit_probing/blob/HEAD/src/circuit_probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55844d9233f79937"}},{"code_sha256_prefix":"fd76c5e635c1455a","entry":"loss_fn","repo":"mlepori1/circuit_probing","repo_kind":"official","path":"src/train_algorithmic_models.py","file_url":"https://github.com/mlepori1/circuit_probing/blob/HEAD/src/train_algorithmic_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd76c5e635c1455a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}