{"url":"/method/patch-merger","slug":"patch-merger","name":"Patch Merger","full_name":"Patch Merger Module","full_name_withheld":false,"description_markdown":"PatchMerger is a module for Vision Transformers that decreases the number of tokens/patches passed onto each individual transformer encoder block whilst maintaining performance and reducing compute. PatchMerger takes linearly transforms an input of shape N patches × D dimensions through a learnable weight matrix of shape M output patches × D. This generates M scores, in which a Softmax function is applied for each score. The resulting output has a shape of M × N, which is multiplied to the original input to get an output of shape M × D.\r\n\r\nMathematically, $$Y = \\text{softmax}({W^T}{X^T})X$$\r\n\r\nImage and formula from: Renggli, C., Pinto, A. S., Houlsby, N., Mustafa, B., Puigcerver, J., & Riquelme, C. (2022). Learning to Merge Tokens in Vision Transformers. arXiv preprint arXiv:2202.12015.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Learning to Merge Tokens in Vision Transformers","paper":"/paper/learning-to-merge-tokens-in-vision","first_author":"Cedric Renggli","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/learning-to-merge-tokens-in-vision"},"source":{"url":"https://arxiv.org/abs/2202.12015v1","title":"Learning to Merge Tokens in Vision Transformers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Image Model Blocks","url":"/methods/category/image-model-blocks","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/learning-to-merge-tokens-in-vision","title":"Learning to Merge Tokens in Vision Transformers","date":"2022-02-24","arxiv_id":"2202.12015","n_code_links":1,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":0}}],"papers_shown":1,"tasks":[{"task":"/task/natural-language-understanding","name":"Natural Language Understanding","papers":1}],"tasks_shown":1,"n_tasks":1,"usage_by_year":[{"year":"2022","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/patch-merger"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}