{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2503-00089","title":"Protein Structure Tokenization: Benchmarking and New Recipe","arxiv_id":"2503.00089","date":"2025-02-28","proceeding":null,"authors":["Xinyu Yuan","Zichen Wang","Marcus Collins","Huzefa Rangwala"],"abstract":"Recent years have witnessed a surge in the development of protein structural tokenization methods, which chunk protein 3D structures into discrete or continuous representations. Structure tokenization enables the direct application of powerful techniques like language modeling for protein structures, and large multimodal models to integrate structures with protein sequences and functional texts. Despite the progress, the capabilities and limitations of these methods remain poorly understood due to the lack of a unified evaluation framework. We first introduce StructTokenBench, a framework that comprehensively evaluates the quality and efficiency of structure tokenizers, focusing on fine-grained local substructures rather than global structures, as typical in existing benchmarks. Our evaluations reveal that no single model dominates all benchmarking perspectives. Observations of codebook under-utilization led us to develop AminoAseed, a simple yet effective strategy that enhances codebook gradient updates and optimally balances codebook size and dimension for improved tokenizer utilization and quality. Compared to the leading model ESM3, our method achieves an average of 6.31% performance improvement across 24 supervised tasks, with sensitivity and utilization rates increased by 12.83% and 124.03%, respectively. Source code and model weights are available at https://github.com/KatarinaYuan/StructTokenBench","url_abs":"https://arxiv.org/abs/2503.00089v2","url_pdf":"https://arxiv.org/pdf/2503.00089v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"2503-00089","repo_url":"https://github.com/katarinayuan/structtokenbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.00089","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.00089"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/katarinayuan/structtokenbench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":9},"by_repo_kind":{"official":{"samples":10,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cc287b7f78ecb5e5","entry":"parse_fasta","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/baselines/protein_mpnn_utils.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/baselines/protein_mpnn_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc287b7f78ecb5e5"}},{"code_sha256_prefix":"ac939bc0a0113658","entry":"batched_gather","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/vqvae_model.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/vqvae_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac939bc0a0113658"}},{"code_sha256_prefix":"497d53eb6d9adbaf","entry":"calculate_binary_clf_metric","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"497d53eb6d9adbaf"}},{"code_sha256_prefix":"bc2286f6f71a2a26","entry":"calculate_multiclass_clf_metric","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc2286f6f71a2a26"}},{"code_sha256_prefix":"0e76d1d950c3cfd5","entry":"calculate_regression_metric","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e76d1d950c3cfd5"}},{"code_sha256_prefix":"17da71765fb0cdf2","entry":"get_tokenizer_device","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/data_module.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/data_module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17da71765fb0cdf2"}},{"code_sha256_prefix":"65c9e7a4b8a02ce8","entry":"node_gather","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/vqvae_model.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/vqvae_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65c9e7a4b8a02ce8"}},{"code_sha256_prefix":"ad6b51206d2729dc","entry":"parse_PDB","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/baselines/protein_mpnn_utils.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/baselines/protein_mpnn_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad6b51206d2729dc"}},{"code_sha256_prefix":"eb6322985d026288","entry":"parse_PDB_biounits","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/baselines/protein_mpnn_utils.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/baselines/protein_mpnn_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb6322985d026288"}},{"code_sha256_prefix":"a35e139460afb244","entry":"three_to_one","repo":"katarinayuan/structtokenbench","repo_kind":"official","path":"src/baselines/protein_mpnn_cif_parser.py","file_url":"https://github.com/katarinayuan/structtokenbench/blob/HEAD/src/baselines/protein_mpnn_cif_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a35e139460afb244"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}