{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ppt-token-pruning-and-pooling-for-efficient","title":"PPT: Token Pruning and Pooling for Efficient Vision Transformers","arxiv_id":"2310.01812","date":"2023-10-03","proceeding":null,"authors":["Xinjian Wu","Fanhu Zeng","Xiudong Wang","Xinghao Chen"],"abstract":"Vision Transformers (ViTs) have emerged as powerful models in the field of computer vision, delivering superior performance across various vision tasks. However, the high computational complexity poses a significant barrier to their practical applications in real-world scenarios. Motivated by the fact that not all tokens contribute equally to the final predictions and fewer tokens bring less computational cost, reducing redundant tokens has become a prevailing paradigm for accelerating vision transformers. However, we argue that it is not optimal to either only reduce inattentive redundancy by token pruning, or only reduce duplicative redundancy by token merging. To this end, in this paper we propose a novel acceleration framework, namely token Pruning & Pooling Transformers (PPT), to adaptively tackle these two types of redundancy in different layers. By heuristically integrating both token pruning and token pooling techniques in ViTs without additional trainable parameters, PPT effectively reduces the model complexity while maintaining its predictive accuracy. For example, PPT reduces over 37% FLOPs and improves the throughput by over 45% for DeiT-S without any accuracy drop on the ImageNet dataset. The code is available at https://github.com/xjwu1024/PPT and https://github.com/mindspore-lab/models/","url_abs":"https://arxiv.org/abs/2310.01812v3","url_pdf":"https://arxiv.org/pdf/2310.01812v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ppt-token-pruning-and-pooling-for-efficient","repo_url":"https://github.com/xjwu1024/PPT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"ppt-token-pruning-and-pooling-for-efficient","repo_url":"https://github.com/mindspore-lab/models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"tasks":[{"task_slug":"efficient-vits","task_name":"Efficient ViTs"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-lv-vit-s","task":"Efficient ViTs","dataset":"ImageNet-1K (With LV-ViT-S)","model":"PPT","rank_in_archive_order":8,"of":19,"metrics":{"GFLOPs":"4.6","Top 1 Accuracy":"83.1"},"uses_additional_data":false},{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-deit-s","task":"Efficient ViTs","dataset":"ImageNet-1K (with DeiT-S)","model":"PPT","rank_in_archive_order":5,"of":41,"metrics":{"GFLOPs":"2.9","Top 1 Accuracy":"79.8"},"uses_additional_data":false},{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-deit-t","task":"Efficient ViTs","dataset":"ImageNet-1K (with DeiT-T)","model":"PPT","rank_in_archive_order":9,"of":22,"metrics":{"GFLOPs":"0.8","Top 1 Accuracy":"72.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01812","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01812"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xjwu1024/PPT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-lab/models","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran":2,"ran_fixture":1,"ran_draft_wrong":1,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"225f430160e1b0f3","entry":"batch_index_select","repo":"xjwu1024/PPT","repo_kind":"official","path":"ppt_deit.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/ppt_deit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"225f430160e1b0f3"}},{"code_sha256_prefix":"424049f15b90678d","entry":"bipartite_soft_matching","repo":"xjwu1024/PPT","repo_kind":"official","path":"merge.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/merge.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"424049f15b90678d"}},{"code_sha256_prefix":"ba6aa5f07daca9cd","entry":"conv3x3","repo":"xjwu1024/PPT","repo_kind":"official","path":"patchconvnet_models.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/patchconvnet_models.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba6aa5f07daca9cd"}},{"code_sha256_prefix":"969e8250d805a154","entry":"do_nothing","repo":"xjwu1024/PPT","repo_kind":"official","path":"merge.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/merge.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"969e8250d805a154"}},{"code_sha256_prefix":"674be50d432e8846","entry":"kth_bipartite_soft_matching","repo":"xjwu1024/PPT","repo_kind":"official","path":"merge.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/merge.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"674be50d432e8846"}},{"code_sha256_prefix":"d58967bba23da4e6","entry":"build_dataset","repo":"xjwu1024/PPT","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d58967bba23da4e6"}},{"code_sha256_prefix":"ebeb7b6e20a434cf","entry":"build_transform","repo":"xjwu1024/PPT","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/xjwu1024/PPT/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ebeb7b6e20a434cf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}