{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/detecting-modularity-in-deep-neural-networks-1","title":"Quantifying Local Specialization in Deep Neural Networks","arxiv_id":"2110.08058","date":"2021-10-13","proceeding":null,"authors":["Shlomi Hod","Daniel Filan","Stephen Casper","Andrew Critch","Stuart Russell"],"abstract":"A neural network is locally specialized to the extent that parts of its computational graph (i.e. structure) can be abstractly represented as performing some comprehensible sub-task relevant to the overall task (i.e. functionality). Are modern deep neural networks locally specialized? How can this be quantified? In this paper, we consider the problem of taking a neural network whose neurons are partitioned into clusters, and quantifying how functionally specialized the clusters are. We propose two proxies for this: importance, which reflects how crucial sets of neurons are to network performance; and coherence, which reflects how consistently their neurons associate with features of the inputs. To measure these proxies, we develop a set of statistical methods based on techniques conventionally used to interpret individual neurons. We apply the proxies to partitionings generated by spectrally clustering a graph representation of the network's neurons with edges determined either by network weights or correlations of activations. We show that these partitionings, even ones based only on weights (i.e. strictly from non-runtime analysis), reveal groups of neurons that are important and coherent. These results suggest that graph-based partitioning can reveal local specialization and that statistical methods can be used to automatedly screen for sets of neurons that can be understood abstractly.","url_abs":"https://arxiv.org/abs/2110.08058v2","url_pdf":"https://arxiv.org/pdf/2110.08058v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"detecting-modularity-in-deep-neural-networks-1","repo_url":"https://github.com/thestephencasper/detecting_nn_modularity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"detecting-modularity-in-deep-neural-networks-1","repo_url":"https://github.com/thestephencasper/local_specialization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.08058","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.08058"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thestephencasper/local_specialization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thestephencasper/detecting_nn_modularity","reach":{"status":"ok"}}],"summary":{"ran_fixture":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"ae0a8fac72ed4de2","entry":"align_stochastically_bipartite_labels","repo":"thestephencasper/local_specialization","repo_kind":"official","path":"src/local_clustering.py","file_url":"https://github.com/thestephencasper/local_specialization/blob/HEAD/src/local_clustering.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae0a8fac72ed4de2"}},{"code_sha256_prefix":"fb7c432644d90803","entry":"mlp_int_to_tup","repo":"thestephencasper/local_specialization","repo_kind":"official","path":"src/spectral_cluster_model.py","file_url":"https://github.com/thestephencasper/local_specialization/blob/HEAD/src/spectral_cluster_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb7c432644d90803"}},{"code_sha256_prefix":"29838f56834eaeb7","entry":"mlp_tup_to_int","repo":"thestephencasper/local_specialization","repo_kind":"official","path":"src/spectral_cluster_model.py","file_url":"https://github.com/thestephencasper/local_specialization/blob/HEAD/src/spectral_cluster_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29838f56834eaeb7"}},{"code_sha256_prefix":"fc28a0d4ecb16bdf","entry":"cnn_tup_to_int","repo":"thestephencasper/local_specialization","repo_kind":"official","path":"src/spectral_cluster_model.py","file_url":"https://github.com/thestephencasper/local_specialization/blob/HEAD/src/spectral_cluster_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc28a0d4ecb16bdf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}