{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-15720","title":"ToaSt: Token Channel Selection and Structured Pruning for Efficient ViT","arxiv_id":"2602.15720","date":"2026-02-17","proceeding":"ICML","authors":["Hyunchan Moon","Cheonjun Park","Steven L. Waslander"],"abstract":"Vision Transformers (ViTs) have achieved remarkable success across various vision tasks, yet their deployment is often hindered by prohibitive computational costs. While structured weight pruning and token compression have emerged as promising solutions, they suffer from prolonged retraining and inter-layer dependencies that complicate optimization, respectively. We propose ToaSt, a decoupled framework applying specialized strategies to distinct ViT components. We apply coupled head-wise structured pruning to Multi-Head Self-Attention modules, leveraging attention operation characteristics to enhance robustness. For Feed-Forward Networks (over 60% of FLOPs), we introduce Token Channel Selection (TCS), a training-free method that filters redundant noise channels at inference time. Extensive evaluations across nine diverse models, including DeiT, ViT-MAE, and Swin Transformer, demonstrate that ToaSt achieves superior trade-offs between accuracy and efficiency, consistently outperforming existing baselines. On ViT-MAE-Huge, ToaSt achieves 88.52% accuracy (+1.64%p) with 39.4% FLOPs reduction. ToaSt also transfers effectively to diverse downstream tasks (COCO detection, ADE20K segmentation, CIFAR-100 classification), achieving 52.2 versus 51.9 mAP on COCO. Code: github.com/SHANNonLab-HUFS/ToaSt","url_abs":"https://arxiv.org/abs/2602.15720","url_pdf":"https://arxiv.org/pdf/2602.15720","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.15720","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.15720"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/SHANNonLab-HUFS/ToaSt","reach":null}],"summary":{"ran_draft_wrong":3,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"51e9315f719ad241","entry":"arch_of","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"51e9315f719ad241"}},{"code_sha256_prefix":"4e7d61d7bca89620","entry":"attention_layers","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4e7d61d7bca89620"}},{"code_sha256_prefix":"2429301e773af19e","entry":"iter_blocks","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2429301e773af19e"}},{"code_sha256_prefix":"d3f3fbc8697f8512","entry":"StructuredCoupledPruner","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3f3fbc8697f8512"}},{"code_sha256_prefix":"6e446cfcdafce8e6","entry":"head_importance","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6e446cfcdafce8e6"}},{"code_sha256_prefix":"ed8687c006a86f0c","entry":"reapply_masks","repo":"SHANNonLab-HUFS/ToaSt","repo_kind":"found_in_text","path":"toast/weight_pruning.py","file_url":"https://github.com/SHANNonLab-HUFS/ToaSt/blob/HEAD/toast/weight_pruning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ed8687c006a86f0c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}