{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-geometry-of-concepts-sparse-autoencoder","title":"The Geometry of Concepts: Sparse Autoencoder Feature Structure","arxiv_id":"2410.19750","date":"2024-10-10","proceeding":null,"authors":["Yuxiao Li","Eric J. Michaud","David D. Baek","Joshua Engels","Xiaoqing Sun","Max Tegmark"],"abstract":"Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: 1) The \"atomic\" small-scale structure contains \"crystals\" whose faces are parallelograms or trapezoids, generalizing well-known examples such as (man-woman-king-queen). We find that the quality of such parallelograms and associated function vectors improves greatly when projecting out global distractor directions such as word length, which is efficiently done with linear discriminant analysis. 2) The \"brain\" intermediate-scale structure has significant spatial modularity; for example, math and code features form a \"lobe\" akin to functional lobes seen in neural fMRI images. We quantify the spatial locality of these lobes with multiple metrics and find that clusters of co-occurring features, at coarse enough scale, also cluster together spatially far more than one would expect if feature geometry were random. 3) The \"galaxy\" scale large-scale structure of the feature point cloud is not isotropic, but instead has a power law of eigenvalues with steepest slope in middle layers. We also quantify how the clustering entropy depends on the layer.","url_abs":"https://arxiv.org/abs/2410.19750v2","url_pdf":"https://arxiv.org/pdf/2410.19750v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-geometry-of-concepts-sparse-autoencoder","repo_url":"https://github.com/ejmichaud/feature-geometry","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"math","task_name":"Math"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.19750","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.19750"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ejmichaud/feature-geometry","reach":{"status":"ok"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"02cad154e3ffa4a0","entry":"binary_entropy","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_mi.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_mi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02cad154e3ffa4a0"}},{"code_sha256_prefix":"e0b480e81fb19669","entry":"generate_output_filename","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e0b480e81fb19669"}},{"code_sha256_prefix":"99993387d21b38f2","entry":"generate_output_filename","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_ang.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_ang.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99993387d21b38f2"}},{"code_sha256_prefix":"9e316f40a6344d09","entry":"get_phi","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_phi.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_phi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e316f40a6344d09"}},{"code_sha256_prefix":"f11fb3e9c0f0e059","entry":"load_W_dec","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_ang.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_ang.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f11fb3e9c0f0e059"}},{"code_sha256_prefix":"81dcb05ee3b4d5b7","entry":"load_W_enc","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_enc_ang.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_enc_ang.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"81dcb05ee3b4d5b7"}},{"code_sha256_prefix":"f9267d46915610da","entry":"load_matrix","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9267d46915610da"}},{"code_sha256_prefix":"4f4c0aaa95ab39ff","entry":"perform_spectral_clustering","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f4c0aaa95ab39ff"}},{"code_sha256_prefix":"d4c38cae8c60b17c","entry":"perform_spectral_clustering","repo":"ejmichaud/feature-geometry","repo_kind":"official","path":"brain/experiments/all-occurrences0/cluster_ang.py","file_url":"https://github.com/ejmichaud/feature-geometry/blob/HEAD/brain/experiments/all-occurrences0/cluster_ang.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4c38cae8c60b17c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}