{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unitok-a-unified-tokenizer-for-visual","title":"UniTok: A Unified Tokenizer for Visual Generation and Understanding","arxiv_id":"2502.20321","date":"2025-02-27","proceeding":null,"authors":["Chuofan Ma","Yi Jiang","Junfeng Wu","Jihan Yang","Xin Yu","Zehuan Yuan","Bingyue Peng","Xiaojuan Qi"],"abstract":"Visual generative and understanding models typically rely on distinct tokenizers to process images, presenting a key challenge for unifying them within a single framework. Recent studies attempt to address this by connecting the training of VQVAE (for autoregressive generation) and CLIP (for understanding) to build a unified tokenizer. However, directly combining these training objectives has been observed to cause severe loss conflicts. In this paper, we show that reconstruction and semantic supervision do not inherently conflict. Instead, the underlying bottleneck stems from limited representational capacity of discrete token space. Building on these insights, we introduce UniTok, a unified tokenizer featuring a novel multi-codebook quantization mechanism that effectively scales up the vocabulary size and bottleneck dimension. In terms of final performance, UniTok sets a new record of 0.38 rFID and 78.6% zero-shot accuracy on ImageNet. Besides, UniTok can be seamlessly integrated into MLLMs to unlock native visual generation capability, without compromising the understanding performance. Additionally, we show that UniTok favors cfg-free generation, reducing gFID from 14.6 to 2.5 on ImageNet 256$\\times$256 benchmark. GitHub: https://github.com/FoundationVision/UniTok.","url_abs":"https://arxiv.org/abs/2502.20321v2","url_pdf":"https://arxiv.org/pdf/2502.20321v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unitok-a-unified-tokenizer-for-visual","repo_url":"https://github.com/foundationvision/unitok","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.20321","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.20321"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/FoundationVision/UniTok","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/foundationvision/unitok","reach":null}],"summary":{"ran":9,"ran_draft_wrong":3,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"134e8fce743a76a0","entry":"AttnProjection","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"134e8fce743a76a0"}},{"code_sha256_prefix":"1d52d9cc8d6405fe","entry":"GeGluMlp","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d52d9cc8d6405fe"}},{"code_sha256_prefix":"925747adc8e48ca7","entry":"InvMbConvLNBlock","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"925747adc8e48ca7"}},{"code_sha256_prefix":"04d969cadbc4e6f5","entry":"InvStem","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04d969cadbc4e6f5"}},{"code_sha256_prefix":"c7585f75fa769b15","entry":"NormalizedEmbedding","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7585f75fa769b15"}},{"code_sha256_prefix":"329e243c3200b380","entry":"PlainAttention","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"329e243c3200b380"}},{"code_sha256_prefix":"aadb57a18aa01718","entry":"Upsample2d","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aadb57a18aa01718"}},{"code_sha256_prefix":"150dd2792a2bcac8","entry":"VectorQuantizer","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"150dd2792a2bcac8"}},{"code_sha256_prefix":"3ce435874422947f","entry":"VectorQuantizerM","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ce435874422947f"}},{"code_sha256_prefix":"ba65976c1d96a7cb","entry":"creat_optimizer","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"ba65976c1d96a7cb"}},{"code_sha256_prefix":"a8580707adda13dd","entry":"get_cosine_scheduler","repo":"FoundationVision/UniTok","repo_kind":"official","path":"eval/llamagen/autoregressive/train/train_c2i.py","file_url":"https://github.com/FoundationVision/UniTok/blob/HEAD/eval/llamagen/autoregressive/train/train_c2i.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8580707adda13dd"}},{"code_sha256_prefix":"b9ad16069d39de05","entry":"get_entropy_loss","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b9ad16069d39de05"}},{"code_sha256_prefix":"0201eaaed93053b2","entry":"UniTok","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0201eaaed93053b2"}},{"code_sha256_prefix":"694e2f6925e2eea4","entry":"ViTaminDecoder","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"694e2f6925e2eea4"}},{"code_sha256_prefix":"a0df8fea0fc36b20","entry":"_init_conv","repo":"foundationvision/unitok","repo_kind":"official","path":"models/unitok.py","file_url":"https://github.com/foundationvision/unitok/blob/HEAD/models/unitok.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0df8fea0fc36b20"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}