{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-heuristic-core-understanding-subnetwork","title":"The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models","arxiv_id":"2403.03942","date":"2024-03-06","proceeding":null,"authors":["Adithya Bhaskar","Dan Friedman","Danqi Chen"],"abstract":"Prior work has found that pretrained language models (LMs) fine-tuned with different random seeds can achieve similar in-domain performance but generalize differently on tests of syntactic generalization. In this work, we show that, even within a single model, we can find multiple subnetworks that perform similarly in-domain, but generalize vastly differently. To better understand these phenomena, we investigate if they can be understood in terms of \"competing subnetworks\": the model initially represents a variety of distinct algorithms, corresponding to different subnetworks, and generalization occurs when it ultimately converges to one. This explanation has been used to account for generalization in simple algorithmic tasks (\"grokking\"). Instead of finding competing subnetworks, we find that all subnetworks -- whether they generalize or not -- share a set of attention heads, which we refer to as the heuristic core. Further analysis suggests that these attention heads emerge early in training and compute shallow, non-generalizing features. The model learns to generalize by incorporating additional attention heads, which depend on the outputs of the \"heuristic\" heads to compute higher-level features. Overall, our results offer a more detailed picture of the mechanisms for syntactic generalization in pretrained LMs.","url_abs":"https://arxiv.org/abs/2403.03942v2","url_pdf":"https://arxiv.org/pdf/2403.03942v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-heuristic-core-understanding-subnetwork","repo_url":"https://github.com/princeton-nlp/heuristic-core","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.03942","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.03942"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/princeton-nlp/heuristic-core","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2ee23441731fc70c","entry":"deterministic_z_from_log_alpha","repo":"princeton-nlp/heuristic-core","repo_kind":"official","path":"src/pruning/l0.py","file_url":"https://github.com/princeton-nlp/heuristic-core/blob/HEAD/src/pruning/l0.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ee23441731fc70c"}},{"code_sha256_prefix":"f60c8575e415f9ae","entry":"cdf_stretched_concrete","repo":"princeton-nlp/heuristic-core","repo_kind":"official","path":"src/pruning/l0.py","file_url":"https://github.com/princeton-nlp/heuristic-core/blob/HEAD/src/pruning/l0.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f60c8575e415f9ae"}},{"code_sha256_prefix":"75c010fd45104632","entry":"load_tf_weights_in_bert","repo":"princeton-nlp/heuristic-core","repo_kind":"official","path":"src/pruning/modeling_pert.py","file_url":"https://github.com/princeton-nlp/heuristic-core/blob/HEAD/src/pruning/modeling_pert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75c010fd45104632"}},{"code_sha256_prefix":"07ce21c57ca66278","entry":"sample_z_from_u","repo":"princeton-nlp/heuristic-core","repo_kind":"official","path":"src/pruning/l0.py","file_url":"https://github.com/princeton-nlp/heuristic-core/blob/HEAD/src/pruning/l0.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07ce21c57ca66278"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}