{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/instilling-inductive-biases-with-subnetworks","title":"Instilling Inductive Biases with Subnetworks","arxiv_id":"2310.10899","date":"2023-10-17","proceeding":null,"authors":["Enyan Zhang","Michael A. Lepori","Ellie Pavlick"],"abstract":"Despite the recent success of artificial neural networks on a variety of tasks, we have little knowledge or control over the exact solutions these models implement. Instilling inductive biases -- preferences for some solutions over others -- into these models is one promising path toward understanding and controlling their behavior. Much work has been done to study the inherent inductive biases of models and instill different inductive biases through hand-designed architectures or carefully curated training regimens. In this work, we explore a more mechanistic approach: Subtask Induction. Our method discovers a functional subnetwork that implements a particular subtask within a trained model and uses it to instill inductive biases towards solutions utilizing that subtask. Subtask Induction is flexible and efficient, and we demonstrate its effectiveness with two experiments. First, we show that Subtask Induction significantly reduces the amount of training data required for a model to adopt a specific, generalizable solution to a modular arithmetic task. Second, we demonstrate that Subtask Induction successfully induces a human-like shape bias while increasing data efficiency for convolutional and transformer-based image classification models.","url_abs":"https://arxiv.org/abs/2310.10899v2","url_pdf":"https://arxiv.org/pdf/2310.10899v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"instilling-inductive-biases-with-subnetworks","repo_url":"https://github.com/rock-z/instilling-inductiva-bias","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.10899","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.10899"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rock-z/instilling-inductiva-bias","reach":{"status":"ok"}}],"summary":{"ran":7},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"bd9c5f376c68862c","entry":"collate_fn","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"vision/pooled_imnet_eval.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/vision/pooled_imnet_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bd9c5f376c68862c"}},{"code_sha256_prefix":"437d2d938ea34c0c","entry":"collate_fn","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"vision/resnet_16_class_adapt.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/vision/resnet_16_class_adapt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"437d2d938ea34c0c"}},{"code_sha256_prefix":"8761d7e28afd1067","entry":"compute_metrics","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"vision/resnet_16_class_adapt.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/vision/resnet_16_class_adapt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8761d7e28afd1067"}},{"code_sha256_prefix":"76e5fff22429bad6","entry":"compute_metrics","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"math/ambiguous.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/math/ambiguous.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76e5fff22429bad6"}},{"code_sha256_prefix":"4dd706fcbcf94648","entry":"compute_similarity_target","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"vision/torch_resnet_ablate.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/vision/torch_resnet_ablate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4dd706fcbcf94648"}},{"code_sha256_prefix":"640a1e4e7c2d8811","entry":"filter_to_16_class","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"vision/pooled_imnet_eval.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/vision/pooled_imnet_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"640a1e4e7c2d8811"}},{"code_sha256_prefix":"4093d31d217c9e02","entry":"generate_samples","repo":"rock-z/instilling-inductiva-bias","repo_kind":"official","path":"math/utils.py","file_url":"https://github.com/rock-z/instilling-inductiva-bias/blob/HEAD/math/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4093d31d217c9e02"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}