{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tokenflow-unified-image-tokenizer-for","title":"TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation","arxiv_id":"2412.03069","date":"2024-12-04","proceeding":"CVPR 2025 1","authors":["Liao Qu","Huichao Zhang","Yiheng Liu","Xu Wang","Yi Jiang","Yiming Gao","Hu Ye","Daniel K. Du","Zehuan Yuan","Xinglong Wu"],"abstract":"We present TokenFlow, a novel unified image tokenizer that bridges the long-standing gap between multimodal understanding and generation. Prior research attempt to employ a single reconstruction-targeted Vector Quantization (VQ) encoder for unifying these two tasks. We observe that understanding and generation require fundamentally different granularities of visual information. This leads to a critical trade-off, particularly compromising performance in multimodal understanding tasks. TokenFlow addresses this challenge through an innovative dual-codebook architecture that decouples semantic and pixel-level feature learning while maintaining their alignment via a shared mapping mechanism. This design enables direct access to both high-level semantic representations crucial for understanding tasks and fine-grained visual features essential for generation through shared indices. Our extensive experiments demonstrate TokenFlow's superiority across multiple dimensions. Leveraging TokenFlow, we demonstrate for the first time that discrete visual input can surpass LLaVA-1.5 13B in understanding performance, achieving a 7.2\\% average improvement. For image reconstruction, we achieve a strong FID score of 0.63 at 384*384 resolution. Moreover, TokenFlow establishes state-of-the-art performance in autoregressive image generation with a GenEval score of 0.55 at 256*256 resolution, achieving comparable results to SDXL.","url_abs":"https://arxiv.org/abs/2412.03069v1","url_pdf":"https://arxiv.org/pdf/2412.03069v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tokenflow-unified-image-tokenizer-for","repo_url":"https://github.com/ByteFlow-AI/TokenFlow","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"tokenflow-unified-image-tokenizer-for","repo_url":"https://github.com/ruohaoguo/avis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2412.03069","atlas_url":"https://app.syntology.ai/?focus=2412.03069","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.03069"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ByteFlow-AI/TokenFlow","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruohaoguo/avis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran_violates":1,"unverified":15},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1},"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1712a07966b542ee","entry":"center_crop_arr","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/dataset/augmentation.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1712a07966b542ee"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/discriminator.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/discriminator.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"22a83df891d750c6","entry":"hinge_d_loss","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_loss.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_loss.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"22a83df891d750c6"}},{"code_sha256_prefix":"c8ec5333f4bcb71f","entry":"leaky_relu","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/discriminator.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/discriminator.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8ec5333f4bcb71f"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_model.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"bc2cb481a75c370d","entry":"batch_dice_loss","repo":"ruohaoguo/avis","repo_kind":"listed","path":"avism/modeling/avism_matcher.py","file_url":"https://github.com/ruohaoguo/avis/blob/HEAD/avism/modeling/avism_matcher.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc2cb481a75c370d"}},{"code_sha256_prefix":"1edd24985036b0bf","entry":"batch_sigmoid_ce_loss","repo":"ruohaoguo/avis","repo_kind":"listed","path":"avism/modeling/avism_matcher.py","file_url":"https://github.com/ruohaoguo/avis/blob/HEAD/avism/modeling/avism_matcher.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1edd24985036b0bf"}},{"code_sha256_prefix":"a5dc3fcd09ebc843","entry":"build_coco","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/dataset/coco.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/dataset/coco.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a5dc3fcd09ebc843"}},{"code_sha256_prefix":"5cceaa9d0edbce06","entry":"build_imagenet","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/dataset/imagenet.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/dataset/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5cceaa9d0edbce06"}},{"code_sha256_prefix":"beae755c2ed7e35a","entry":"build_model","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/clip/model.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/clip/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"beae755c2ed7e35a"}},{"code_sha256_prefix":"0a65aaf586d4f90b","entry":"build_openimage","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/dataset/openimage.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/dataset/openimage.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a65aaf586d4f90b"}},{"code_sha256_prefix":"2dcb8123d89bb1ff","entry":"calculate_uncertainty","repo":"ruohaoguo/avis","repo_kind":"listed","path":"avism/modeling/avism_criterion.py","file_url":"https://github.com/ruohaoguo/avis/blob/HEAD/avism/modeling/avism_criterion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2dcb8123d89bb1ff"}},{"code_sha256_prefix":"3980a12a0d103b4f","entry":"clip_model","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/clip/model.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/clip/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3980a12a0d103b4f"}},{"code_sha256_prefix":"797e47600ed8da5f","entry":"copy_new_embedding","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_model.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"797e47600ed8da5f"}},{"code_sha256_prefix":"89f75e54ff128be0","entry":"dice_loss","repo":"ruohaoguo/avis","repo_kind":"listed","path":"avism/modeling/avism_criterion.py","file_url":"https://github.com/ruohaoguo/avis/blob/HEAD/avism/modeling/avism_criterion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89f75e54ff128be0"}},{"code_sha256_prefix":"f3dad8208b724c43","entry":"drop_scale","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_model.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f3dad8208b724c43"}},{"code_sha256_prefix":"dd2278aa669b6313","entry":"non_saturating_d_loss","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_loss.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd2278aa669b6313"}},{"code_sha256_prefix":"980fb099f82d8b84","entry":"random_crop_arr","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/dataset/augmentation.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"980fb099f82d8b84"}},{"code_sha256_prefix":"d0c61e8dba511aa3","entry":"sigmoid_ce_loss","repo":"ruohaoguo/avis","repo_kind":"listed","path":"avism/modeling/avism_criterion.py","file_url":"https://github.com/ruohaoguo/avis/blob/HEAD/avism/modeling/avism_criterion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0c61e8dba511aa3"}},{"code_sha256_prefix":"519cb60c0e3fbcb2","entry":"vanilla_d_loss","repo":"ByteFlow-AI/TokenFlow","repo_kind":"official","path":"tokenflow/tokenizer/vq_loss.py","file_url":"https://github.com/ByteFlow-AI/TokenFlow/blob/HEAD/tokenflow/tokenizer/vq_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"519cb60c0e3fbcb2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}