{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-03328","title":"StructLens: A Structural Lens for Language Models via Maximum Spanning Trees","arxiv_id":"2603.03328","date":"2026-02-10","proceeding":null,"authors":["Haruki Sakajo","Frederikus Hudi","Yusuke Sakai","Hidetaka Kamigaito","Taro Watanabe"],"abstract":"Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest their own internal structures as well. While interpretability research has investigated how models compute representations mechanistically through attention patterns and Sparse AutoEncoders, the organization of the resulting representations is overlooked. To address this gap, we introduce StructLens, a framework to analyze representations through a holistic structural view. StructLens constructs maximum spanning trees based on the semantic representations in residual streams, inspired by tree representation in dependency parsing, and provides summaries of token relationships in representation space. We analyze how contiguous tokens are also nearby in representation space and find that middle layers show the strongest local-span organization. Moreover, analysis of pre-training checkpoints reveals that smaller local units become detectable earlier in pre-training, and larger units later. Our findings demonstrate that StructLens provides insights into how models organize token representations across layers and training. Our code is available at https://github.com/naist-nlp/structlens.","url_abs":"https://arxiv.org/abs/2603.03328","url_pdf":"https://arxiv.org/pdf/2603.03328","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.03328"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/naist-nlp/structlens","reach":null}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"11473ef9ba8fa9dc","entry":"convert_tree_format","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/metrics.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"11473ef9ba8fa9dc"}},{"code_sha256_prefix":"b471fd6c8b4bcea5","entry":"count_subtree_sizes","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/metrics.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b471fd6c8b4bcea5"}},{"code_sha256_prefix":"98938b9ad4dc6e3f","entry":"create_masks","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/masking.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/masking.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98938b9ad4dc6e3f"}},{"code_sha256_prefix":"6d3ecc9fce1c2ffe","entry":"get_logger","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/utils/logging_config.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/utils/logging_config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d3ecc9fce1c2ffe"}},{"code_sha256_prefix":"739cb66d97a8b03c","entry":"l2_distance","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/similarity.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/similarity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"739cb66d97a8b03c"}},{"code_sha256_prefix":"80659e1fbbca9198","entry":"l2_reciprocal_similarity","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/similarity.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/similarity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80659e1fbbca9198"}},{"code_sha256_prefix":"99a847c6af296de3","entry":"parent_array_to_tree","repo":"naist-nlp/structlens","repo_kind":"found_in_text","path":"src/structlens/metrics.py","file_url":"https://github.com/naist-nlp/structlens/blob/HEAD/src/structlens/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99a847c6af296de3"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}