{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-merge-tokens-in-vision","title":"Learning to Merge Tokens in Vision Transformers","arxiv_id":"2202.12015","date":"2022-02-24","proceeding":null,"authors":["Cedric Renggli","André Susano Pinto","Neil Houlsby","Basil Mustafa","Joan Puigcerver","Carlos Riquelme"],"abstract":"Transformers are widely applied to solve natural language understanding and computer vision tasks. While scaling up these architectures leads to improved performance, it often comes at the expense of much higher computational costs. In order for large-scale models to remain practical in real-world systems, there is a need for reducing their computational overhead. In this work, we present the PatchMerger, a simple module that reduces the number of patches or tokens the network has to process by merging them between two consecutive intermediate layers. We show that the PatchMerger achieves a significant speedup across various model sizes while matching the original performance both upstream and downstream after fine-tuning.","url_abs":"https://arxiv.org/abs/2202.12015v1","url_pdf":"https://arxiv.org/pdf/2202.12015v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-merge-tokens-in-vision","repo_url":"https://github.com/conceptofmind/ViT-Patch-Merger","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"}],"methods":[{"method_slug":"patch-merger","method_name":"Patch Merger"}],"datasets_introduced":[],"methods_introduced":[{"slug":"patch-merger","name":"Patch Merger","full_name":"Patch Merger Module"}],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2202.12015","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.12015"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/conceptofmind/ViT-Patch-Merger","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"conceptofmind/ViT-Patch-Merger","repo_kind":"listed","path":"patch_merger.py","file_url":"https://github.com/conceptofmind/ViT-Patch-Merger/blob/HEAD/patch_merger.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"conceptofmind/ViT-Patch-Merger","repo_kind":"listed","path":"patch_merger.py","file_url":"https://github.com/conceptofmind/ViT-Patch-Merger/blob/HEAD/patch_merger.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"6ba8cee9f5daea41","entry":"pair","repo":"conceptofmind/ViT-Patch-Merger","repo_kind":"listed","path":"patch_merger.py","file_url":"https://github.com/conceptofmind/ViT-Patch-Merger/blob/HEAD/patch_merger.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ba8cee9f5daea41"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}