{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/finding-transformer-circuits-with-edge","title":"Finding Transformer Circuits with Edge Pruning","arxiv_id":"2406.16778","date":"2024-06-24","proceeding":null,"authors":["Adithya Bhaskar","Alexander Wettig","Dan Friedman","Danqi Chen"],"abstract":"The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Recent work has automated the task of discovering circuits. Yet, these methods have practical limitations, as they rely either on inefficient search algorithms or inaccurate approximations. In this paper, we frame automated circuit discovery as an optimization problem and propose *Edge Pruning* as an effective and scalable solution. Edge Pruning leverages gradient-based pruning techniques, but instead of removing neurons or components, it prunes the \\emph{edges} between components. Our method finds circuits in GPT-2 that use less than half the number of edges compared to circuits found by previous methods while being equally faithful to the full model predictions on standard circuit-finding tasks. Edge Pruning is efficient even with as many as 100K examples, outperforming previous methods in speed and producing substantially better circuits. It also perfectly recovers the ground-truth circuits in two models compiled with Tracr. Thanks to its efficiency, we scale Edge Pruning to CodeLlama-13B, a model over 100x the scale that prior methods operate on. We use this setting for a case study comparing the mechanisms behind instruction prompting and in-context learning. We find two circuits with more than 99.96% sparsity that match the performance of the full model and reveal that the mechanisms in the two settings overlap substantially. Our case study shows that Edge Pruning is a practical and scalable tool for interpretability and sheds light on behaviors that only emerge in large models.","url_abs":"https://arxiv.org/abs/2406.16778v2","url_pdf":"https://arxiv.org/pdf/2406.16778v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"finding-transformer-circuits-with-edge","repo_url":"https://github.com/princeton-nlp/edge-pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-2","method_name":"GPT-2"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.16778","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.16778"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/princeton-nlp/edge-pruning","reach":null}],"summary":{"ran_draft_wrong":5,"ran_honours":3},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ac94bbb65658849e","entry":"find_template","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/eval/ioi.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/eval/ioi.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac94bbb65658849e"}},{"code_sha256_prefix":"65258336cdb05cba","entry":"format_instance","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/prune/fpt2_ioi.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/prune/fpt2_ioi.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65258336cdb05cba"}},{"code_sha256_prefix":"d4e547744737a080","entry":"n_readers","repo":"princeton-nlp/edge-pruning","repo_kind":"official","path":"src/modeling/modeling_erazr.py","file_url":"https://github.com/princeton-nlp/edge-pruning/blob/HEAD/src/modeling/modeling_erazr.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4e547744737a080"}},{"code_sha256_prefix":"d4f39f78881244a5","entry":"n_total_edges","repo":"princeton-nlp/edge-pruning","repo_kind":"official","path":"src/modeling/modeling_erazr.py","file_url":"https://github.com/princeton-nlp/edge-pruning/blob/HEAD/src/modeling/modeling_erazr.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4f39f78881244a5"}},{"code_sha256_prefix":"e29fd15d44a01690","entry":"reader_idx_to_name","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/modeling/modeling_fpt2.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/modeling/modeling_fpt2.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e29fd15d44a01690"}},{"code_sha256_prefix":"09edbe6bc59ee172","entry":"try_fit_template","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/prune/fpt2_ioi.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/prune/fpt2_ioi.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09edbe6bc59ee172"}},{"code_sha256_prefix":"149ae20dadcb3e8f","entry":"writer_idx_to_name","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/modeling/modeling_fpt2.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/modeling/modeling_fpt2.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"149ae20dadcb3e8f"}},{"code_sha256_prefix":"a8e155123b432821","entry":"writer_name_to_idx","repo":"princeton-nlp/Edge-Pruning","repo_kind":"official","path":"src/modeling/modeling_fpt2.py","file_url":"https://github.com/princeton-nlp/Edge-Pruning/blob/HEAD/src/modeling/modeling_fpt2.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8e155123b432821"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}