{"url":"/method/std","slug":"std","name":"STD","full_name":"Spatial-Channel Token Distillation","full_name_withheld":false,"description_markdown":"The **Spatial-Channel Token Distillation** method is proposed to improve the spatial and channel mixing from a novel knowledge distillation (KD) perspective. To be specific, we design a special KD mechanism for MLP-like Vision Models called Spatial-channel Token Distillation (STD), which improves the information mixing in both the spatial and channel dimensions of MLP blocks. Instead of modifying the mixing operations themselves, STD adds spatial and channel tokens to image patches. After forward propagation, the tokens are concatenated for distillation with the teachers’ responses as targets. Each token works as an aggregator of its dimension. The objective of them is to encourage each mixing operation to extract maximal task-related information from their specific dimension.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Spatial-Channel Token Distillation for Vision MLPs","paper":"/paper/spatial-channel-token-distillation-for-vision","first_author":"Yanxi Li","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/spatial-channel-token-distillation-for-vision"},"source":{"url":"https://proceedings.mlr.press/v162/li22c.html","title":"Spatial-Channel Token Distillation for Vision MLPs","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Knowledge Distillation","url":"/methods/category/knowledge-distillation","pwc_aliases":[]}],"n_papers_tagged":32,"archive_num_papers":32,"papers_newest_first":[{"paper":null,"title":"Geo-Registration of Terrestrial LiDAR Point Clouds with Satellite Images without GNSS","date":"2025-07-08","arxiv_id":"2507.05999","n_code_links":0,"syntology":null},{"paper":null,"title":"Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs","date":"2025-05-25","arxiv_id":"2505.19155","n_code_links":0,"syntology":null},{"paper":"/paper/re-trip-reflectivity-instance-augmented","title":"RE-TRIP : Reflectivity Instance Augmented Triangle Descriptor for 3D Place Recognition","date":"2025-05-22","arxiv_id":"2505.16165","n_code_links":1,"syntology":null},{"paper":null,"title":"A New Paradigm in IBR Modeling for Power Flow and Short Circuit Analysis","date":"2025-04-14","arxiv_id":"2504.10181","n_code_links":0,"syntology":null},{"paper":"/paper/dualquat-loam-lidar-odometry-and-mapping-1","title":"DualQuat-LOAM: LiDAR Odometry and Mapping parameterized on Dual Quaternions","date":"2025-04-07","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects","date":"2025-03-21","arxiv_id":"2503.17106","n_code_links":0,"syntology":null},{"paper":null,"title":"Multimodal AI on Wound Images and Clinical Notes for Home Patient Referral","date":"2025-01-22","arxiv_id":"2501.13247","n_code_links":0,"syntology":null},{"paper":"/paper/mhgnet-multi-heterogeneous-graph-neural","title":"MHGNet: Multi-Heterogeneous Graph Neural Network for Traffic Prediction","date":"2025-01-07","arxiv_id":"2501.03635","n_code_links":1,"syntology":null},{"paper":null,"title":"Leveraging Prompt Learning and Pause Encoding for Alzheimer's Disease Detection","date":"2024-12-09","arxiv_id":"2412.06259","n_code_links":0,"syntology":null},{"paper":"/paper/best-std-bidirectional-mamba-enhanced-speech","title":"BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection","date":"2024-11-21","arxiv_id":"2411.14100","n_code_links":1,"syntology":null},{"paper":null,"title":"Spotlight Text Detector: Spotlight on Candidate Regions Like a Camera","date":"2024-09-25","arxiv_id":"2409.16820","n_code_links":0,"syntology":null},{"paper":"/paper/fairness-under-cover-evaluating-the-impact-of","title":"Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition","date":"2024-08-19","arxiv_id":"2408.10175","n_code_links":1,"syntology":null},{"paper":"/paper/iftd-image-feature-triangle-descriptor-for","title":"IFTD: Image Feature Triangle Descriptor for Loop Detection in Driving Scenes","date":"2024-06-12","arxiv_id":"2406.07937","n_code_links":1,"syntology":null},{"paper":null,"title":"Excess Delay from GDP: Measurement and Causal Analysis","date":"2024-05-18","arxiv_id":"2405.11211","n_code_links":0,"syntology":null},{"paper":null,"title":"Adapting an Artificial Intelligence Sexually Transmitted Diseases Symptom Checker Tool for Mpox Detection: The HeHealth Experience","date":"2024-04-23","arxiv_id":"2404.16885","n_code_links":0,"syntology":null},{"paper":null,"title":"On Extending the Automatic Test Markup Language (ATML) for Machine Learning","date":"2024-04-04","arxiv_id":"2404.03769","n_code_links":0,"syntology":null},{"paper":null,"title":"Disturbance Ratio for Optimal Multi-Event Classification in Power Distribution Networks","date":"2024-02-18","arxiv_id":"2402.11668","n_code_links":0,"syntology":null},{"paper":null,"title":"Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN","date":"2024-02-03","arxiv_id":"2402.02262","n_code_links":0,"syntology":null},{"paper":null,"title":"Bootstrap Your Own Variance","date":"2023-12-06","arxiv_id":"2312.03213","n_code_links":0,"syntology":null},{"paper":null,"title":"Uncertainty quantification for deep learning-based schemes for solving high-dimensional backward stochastic differential equations","date":"2023-10-05","arxiv_id":"2310.03393","n_code_links":0,"syntology":null},{"paper":"/paper/shadocformer-a-shadow-attentive-threshold","title":"ShaDocFormer: A Shadow-Attentive Threshold Detector With Cascaded Fusion Refiner for Document Shadow Removal","date":"2023-09-13","arxiv_id":"2309.06670","n_code_links":1,"syntology":null},{"paper":"/paper/spatial-transform-decoupling-for-oriented","title":"Spatial Transform Decoupling for Oriented Object Detection","date":"2023-08-21","arxiv_id":"2308.10561","n_code_links":1,"syntology":null},{"paper":"/paper/markerless-motion-tracking-with-noisy-video","title":"Markerless Motion Tracking with Noisy Video and IMU Data","date":"2023-05-12","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Spatiotemporal Regularized Tucker Decomposition Approach for Traffic Data Imputation","date":"2023-05-11","arxiv_id":"2305.06563","n_code_links":0,"syntology":null},{"paper":null,"title":"Transformer-based encoder-encoder architecture for Spoken Term Detection","date":"2022-11-02","arxiv_id":"2211.01089","n_code_links":0,"syntology":null},{"paper":"/paper/exploiting-prompt-learning-with-pre-trained","title":"Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection","date":"2022-10-29","arxiv_id":"2210.16539","n_code_links":1,"syntology":null},{"paper":null,"title":"Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddings","date":"2022-10-21","arxiv_id":"2210.11895","n_code_links":0,"syntology":null},{"paper":"/paper/std-stable-triangle-descriptor-for-3d-place","title":"STD: Stable Triangle Descriptor for 3D place recognition","date":"2022-09-26","arxiv_id":"2209.12435","n_code_links":1,"syntology":null},{"paper":null,"title":"Partial annotations for the segmentation of large structures with low annotation cost","date":"2022-09-25","arxiv_id":"2209.12216","n_code_links":0,"syntology":null},{"paper":null,"title":"Text Growing on Leaf","date":"2022-09-07","arxiv_id":"2209.03016","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/pose-estimation","name":"Pose Estimation","papers":3},{"task":"/task/3d-place-recognition","name":"3D Place Recognition","papers":2},{"task":"/task/alzheimer-s-disease-detection","name":"Alzheimer's Disease Detection","papers":2},{"task":"/task/language-modelling","name":"Language Modelling","papers":2},{"task":"/task/prompt-learning","name":"Prompt Learning","papers":2},{"task":"/task/scene-text-detection","name":"Scene Text Detection","papers":2},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":2},{"task":"/task/text-detection","name":"Text Detection","papers":2},{"task":"/task/adversarial-robustness","name":"Adversarial Robustness","papers":1},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":1},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/depression-detection","name":"Depression Detection","papers":1},{"task":"/task/depth-completion","name":"Depth Completion","papers":1},{"task":"/task/depth-estimation","name":"Depth Estimation","papers":1},{"task":"/task/depth-prediction","name":"Depth Prediction","papers":1},{"task":"/task/document-shadow-removal","name":"Document Shadow Removal","papers":1},{"task":"/task/drift-detection","name":"Drift Detection","papers":1},{"task":"/task/face-recognition","name":"Face Recognition","papers":1},{"task":"/task/fairness","name":"Fairness","papers":1}],"tasks_shown":20,"n_tasks":56,"usage_by_year":[{"year":"2022","papers":8},{"year":"2023","papers":6},{"year":"2024","papers":10},{"year":"2025","papers":8}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/std"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}