{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/msvit-dynamic-mixed-scale-tokenization-for","title":"MSViT: Dynamic Mixed-Scale Tokenization for Vision Transformers","arxiv_id":"2307.02321","date":"2023-07-05","proceeding":null,"authors":["Jakob Drachmann Havtorn","Amelie Royer","Tijmen Blankevoort","Babak Ehteshami Bejnordi"],"abstract":"The input tokens to Vision Transformers carry little semantic meaning as they are defined as regular equal-sized patches of the input image, regardless of its content. However, processing uniform background areas of an image should not necessitate as much compute as dense, cluttered areas. To address this issue, we propose a dynamic mixed-scale tokenization scheme for ViT, MSViT. Our method introduces a conditional gating mechanism that selects the optimal token scale for every image region, such that the number of tokens is dynamically determined per input. In addition, to enhance the conditional behavior of the gate during training, we introduce a novel generalization of the batch-shaping loss. We show that our gating module is able to learn meaningful semantics despite operating locally at the coarse patch-level. The proposed gating module is lightweight, agnostic to the choice of transformer backbone, and trained within a few epochs with little training overhead. Furthermore, in contrast to token pruning, MSViT does not lose information about the input, thus can be readily applied for dense tasks. We validate MSViT on the tasks of classification and segmentation where it leads to improved accuracy-complexity trade-off.","url_abs":"https://arxiv.org/abs/2307.02321v2","url_pdf":"https://arxiv.org/pdf/2307.02321v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"msvit-dynamic-mixed-scale-tokenization-for","repo_url":"https://github.com/qualcomm-ai-research/batchshaping","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.02321","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.02321"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qualcomm-ai-research/batchshaping","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"65c95846070f4562","entry":"get_normal_cdf","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/cdf_normal.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/cdf_normal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"65c95846070f4562"}},{"code_sha256_prefix":"2ff74f6eac0c1d2e","entry":"get_relaxed_bernoulli_cdf","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/cdf_relaxedbernoulli.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/cdf_relaxedbernoulli.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2ff74f6eac0c1d2e"}},{"code_sha256_prefix":"8f0e248c0dd763e3","entry":"get_relaxed_bernoulli_pdf","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/cdf_relaxedbernoulli.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/cdf_relaxedbernoulli.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8f0e248c0dd763e3"}},{"code_sha256_prefix":"e880f7dc0d93c7c5","entry":"init_anim_plot","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/viz_utils.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/viz_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e880f7dc0d93c7c5"}},{"code_sha256_prefix":"dc04c9c7a65027eb","entry":"init_gbas_anim_plot","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/viz_utils.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/viz_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"dc04c9c7a65027eb"}},{"code_sha256_prefix":"1c78c301af20fc6f","entry":"to_numpy","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/utils.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1c78c301af20fc6f"}},{"code_sha256_prefix":"737717d0bd22c5fb","entry":"validate_prior_param","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/utils.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"737717d0bd22c5fb"}},{"code_sha256_prefix":"e9256d0acc5a36e8","entry":"warmup_factory","repo":"qualcomm-ai-research/batchshaping","repo_kind":"official","path":"batchshaping/utils.py","file_url":"https://github.com/qualcomm-ai-research/batchshaping/blob/HEAD/batchshaping/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e9256d0acc5a36e8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}