Methods › Computer Vision › Image Models › DeiT
Data-efficient Image Transformer
DeiT
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A Data-Efficient Image Transformer is a type of Vision Transformer for image classification tasks. The model is trained using a teacher-student strategy specific to transformers. It relies on a distillation token ensuring that the student learns from the teacher through attention.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: DeiT-LT.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 93 · with a code link: 60 · where Syntology ran a sample: 27 (22 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (27 of 93 tagged: 22 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 93 of 93, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
DART: Differentiable Dynamic Adaptive Region Tokenizer for Vision Transformer and Mamba 12 Jun 2025 · 1 repository · arXiv:2506.10390
-
Is Attention Required for Transformer Inference? Explore Function-preserving Attention Replacement 24 May 2025 · 0 repositories · arXiv:2505.21535
-
Deep learning-based identification of precipitation clouds from all-sky camera data for observatory safety 24 Mar 2025 · 0 repositories · arXiv:2503.18670
-
CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features 15 Feb 2025 · 0 repositories · arXiv:2502.10682
-
Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity 10 Jan 2025 · 0 repositories · arXiv:2501.06357
-
Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiency 1 Jan 2025 · 0 repositories
-
Training Noise Token Pruning 27 Nov 2024 · 1 repository · arXiv:2411.18092
-
Scalable iterative pruning of large language and vision models using block coordinate descent 26 Nov 2024 · 0 repositories · arXiv:2411.17796
-
PQV-Mobile: A Combined Pruning and Quantization Toolkit to Optimize Vision Transformers for Mobile Applications 15 Aug 2024 · 1 repository · arXiv:2408.08437
-
Mixed Non-linear Quantization for Vision Transformers 26 Jul 2024 · 1 repository · arXiv:2407.18437
-
Knowledge distillation to effectively attain both region-of-interest and global semantics from an image where multiple objects appear 11 Jul 2024 · 1 repository · arXiv:2407.08257
-
Leveraging Transformers for Weakly Supervised Object Localization in Unconstrained Videos 8 Jul 2024 · 1 repository · arXiv:2407.06018
-
Predicting Depression and Anxiety Risk in Dutch Neighborhoods from Street-View Images 27 Jun 2024 · 0 repositories · arXiv:2407.09547
-
LieRE: Generalizing Rotary Position Encodings 14 Jun 2024 · 1 repository · arXiv:2406.10322Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)
-
Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP 3 Jun 2024 · 1 repository · arXiv:2406.01583Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 1 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 13 harvested samples) · 13 pointer-only (licence)
-
A General and Efficient Training for Transformer via Token Expansion 31 Mar 2024 · 1 repository · arXiv:2404.00672Syntology official (archive's flag): 7 ran · 7 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)
-
Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers 15 Mar 2024 · 1 repository · arXiv:2403.10030Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples)
-
MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer 26 Jan 2024 · 0 repositories · arXiv:2401.14895
-
Detecting and recognizing characters in Greek papyri with YOLOv8, DeiT and SimCLR 23 Jan 2024 · 0 repositories · arXiv:2401.12513
-
LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation 20 Jan 2024 · 0 repositories · arXiv:2401.11243
-
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model 17 Jan 2024 · 15 repositories · arXiv:2401.09417Syntology official: no sample here; runs from other or unrecorded repositories · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
TPC-ViT: Token Propagation Controller for Efficient Vision Transformer 3 Jan 2024 · 0 repositories · arXiv:2401.01470
-
ROI-Aware Multiscale Cross-Attention Vision Transformer for Pest Image Identification 28 Dec 2023 · 0 repositories · arXiv:2312.16914
-
Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning 26 Oct 2023 · 0 repositories · arXiv:2310.17177
-
USDC: Unified Static and Dynamic Compression for Visual Transformer 17 Oct 2023 · 0 repositories · arXiv:2310.11117
-
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns 11 Oct 2023 · 0 repositories · arXiv:2310.07664
-
EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification 28 Sep 2023 · 0 repositories · arXiv:2311.12823
-
Multi-Dimensional Hyena for Spatial Inductive Bias 24 Sep 2023 · 0 repositories · arXiv:2309.13600
-
Weight Averaging Improves Knowledge Distillation under Domain Shift 20 Sep 2023 · 1 repository · arXiv:2309.11446
-
Tuning Pre-trained Model via Moment Probing 21 Jul 2023 · 1 repository · arXiv:2307.11342Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 6 where Syntology's instrument failed) · 5 unverified (of 13 harvested samples) · 13 pointer-only (licence)
-
Domain Generalisation with Bidirectional Encoder Representations from Vision Transformers 16 Jul 2023 · 0 repositories · arXiv:2307.08117
-
BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Models 29 Jun 2023 · 1 repository · arXiv:2306.16678
-
Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech 25 May 2023 · 0 repositories · arXiv:2305.16406
-
Bi-ViT: Pushing the Limit of Vision Transformer Quantization 21 May 2023 · 0 repositories · arXiv:2305.12354
-
What Makes for Good Visual Tokenizers for Large Language Models? 20 May 2023 · 1 repository · arXiv:2305.12223Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
Instance-Conditioned GAN Data Augmentation for Representation Learning 16 Mar 2023 · 0 repositories · arXiv:2303.09677
-
X-Pruner: eXplainable Pruning for Vision Transformers 8 Mar 2023 · 1 repository · arXiv:2303.04935
-
Training-Free Acceleration of ViTs with Delayed Spatial Merging 4 Mar 2023 · 1 repository · arXiv:2303.02331
-
A Simple Recipe for Competitive Low-compute Self supervised Vision Models 23 Jan 2023 · 0 repositories · arXiv:2301.09451
-
Detecting Severity of Diabetic Retinopathy from Fundus Images: A Transformer Network-based Review 3 Jan 2023 · 0 repositories · arXiv:2301.00973
-
DropKey for Vision Transformer 1 Jan 2023 · 0 repositories
-
Improving CLIP Fine-tuning Performance 1 Jan 2023 · 1 repository
-
Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint 18 Dec 2022 · 1 repository · arXiv:2212.09062Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
What do Vision Transformers Learn? A Visual Exploration 13 Dec 2022 · 1 repository · arXiv:2212.06727
-
Position Embedding Needs an Independent Layer Normalization 10 Dec 2022 · 1 repository · arXiv:2212.05262
-
NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers 29 Nov 2022 · 1 repository · arXiv:2211.16056Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Model 23 Nov 2022 · 0 repositories · arXiv:2211.12944
-
EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones 17 Nov 2022 · 1 repository · arXiv:2211.09703
-
Data Level Lottery Ticket Hypothesis for Vision Transformers 2 Nov 2022 · 1 repository · arXiv:2211.01484
-
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models 14 Oct 2022 · 0 repositories · arXiv:2210.09223
-
Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer 13 Oct 2022 · 1 repository · arXiv:2210.06707
-
Locality Guidance for Improving Vision Transformers on Tiny Datasets 20 Jul 2022 · 1 repository · arXiv:2207.10026Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
Lightweight Vision Transformer with Cross Feature Attention 15 Jul 2022 · 0 repositories · arXiv:2207.07268
-
Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment 28 Jun 2022 · 1 repository · arXiv:2206.13951Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
SimA: Simple Softmax-free Attention for Vision Transformers 17 Jun 2022 · 1 repository · arXiv:2206.08898Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Sports Re-ID: Improving Re-Identification Of Players In Broadcast Videos Of Team Sports 6 Jun 2022 · 1 repository · arXiv:2206.02373
-
Automated Progressive Learning for Efficient Training of Vision Transformers 28 Mar 2022 · 2 repositories · arXiv:2203.14509Syntology official (archive's flag): 7 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 13 harvested samples) · 4 pointer-only (licence)
-
Unified Visual Transformer Compression 15 Mar 2022 · 1 repository · arXiv:2203.08243Syntology official (archive's flag): 7 ran · 9 ran (of which 3 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 7 unverified (of 16 harvested samples) · 3 pointer-only (licence)
-
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy 12 Mar 2022 · 1 repository · arXiv:2203.06345Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
DiT: Self-supervised Pre-training for Document Image Transformer 4 Mar 2022 · 4 repositories · arXiv:2203.02378Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples)
-
Ensembles of Vision Transformers as a New Paradigm for Automated Classification in Ecology 3 Mar 2022 · 1 repository · arXiv:2203.01726
-
BOAT: Bilateral Local Attention Vision Transformer 31 Jan 2022 · 1 repository · arXiv:2201.13027
-
A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes 26 Jan 2022 · 1 repository · arXiv:2201.10766Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
TerViT: An Efficient Ternary Vision Transformer 20 Jan 2022 · 0 repositories · arXiv:2201.08050
-
Q-ViT: Fully Differentiable Quantization for Vision Transformer 19 Jan 2022 · 1 repository · arXiv:2201.07703Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 9 pointer-only (licence)
-
Multi-Dimensional Model Compression of Vision Transformer 31 Dec 2021 · 1 repository · arXiv:2201.00043
-
Container: Context Aggregation Networks 1 Dec 2021 · 2 repositories
-
Searching the Search Space of Vision Transformer 29 Nov 2021 · 2 repositories · arXiv:2111.14725
-
SWAT: Spatial Structure Within and Among Tokens 26 Nov 2021 · 1 repository · arXiv:2111.13677
-
DBIA: Data-free Backdoor Injection Attack against Transformer Networks 22 Nov 2021 · 1 repository · arXiv:2111.11870
-
Global Vision Transformer Pruning with Hessian-Aware Saliency 10 Oct 2021 · 1 repository · arXiv:2110.04869Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Token Pooling in Vision Transformers 8 Oct 2021 · 0 repositories · arXiv:2110.03860
-
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer 5 Oct 2021 · 31 repositories · arXiv:2110.02178Syntology official: no sample here; runs from other or unrecorded repositories · 53 ran (of which 31 constructed an object rather than computing a result; 44 with no instrument failure: 1 honoured, 0 violated, 43 with no contract checked; 9 where Syntology's instrument failed) · 15 unverified (of 68 harvested samples) · 18 pointer-only (licence)
-
MaiT: integrating spatial locality into image transformers with attention masks 29 Sep 2021 · 1 repository
-
Searching for Efficient Multi-Stage Vision Transformers 1 Sep 2021 · 1 repository · arXiv:2109.00642
-
Exploring and Improving Mobile Level Vision Transformers 30 Aug 2021 · 0 repositories · arXiv:2108.13015
-
PSViT: Better Vision Transformer via Token Pooling and Attention Sharing 7 Aug 2021 · 0 repositories · arXiv:2108.03428
-
Rethinking and Improving Relative Position Encoding for Vision Transformer 29 Jul 2021 · 1 repository · arXiv:2107.14222Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples)
-
A Comparative Study of Deep Learning Classification Methods on a Small Environmental Microorganism Image Dataset (EMDS-6): from Convolutional Neural Networks to Visual Transformers 16 Jul 2021 · 0 repositories · arXiv:2107.07699
-
CMT: Convolutional Neural Networks Meet Vision Transformers 13 Jul 2021 · 14 repositories · arXiv:2107.06263Syntology community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (of 9 harvested samples) · 3 pointer-only (licence)
-
AutoFormer: Searching Transformers for Visual Recognition 1 Jul 2021 · 2 repositories · arXiv:2107.00651
-
ViTAS: Vision Transformer Architecture Search 25 Jun 2021 · 1 repository · arXiv:2106.13700
-
IA-RED²: Interpretability-Aware Redundancy Reduction for Vision Transformers 23 Jun 2021 · 0 repositories · arXiv:2106.12620
-
Solution for Large-scale Long-tailed Recognition with Noisy Labels 20 Jun 2021 · 1 repository · arXiv:2106.10683
-
BEiT: BERT Pre-Training of Image Transformers 15 Jun 2021 · 14 repositories · arXiv:2106.08254Syntology official (archive's flag): 1 ran · 6 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples)
-
Container: Context Aggregation Network 2 Jun 2021 · 4 repositories · arXiv:2106.01401Syntology official (archive's flag): 1 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples)
-
Self-Supervised Learning with Swin Transformers 10 May 2021 · 6 repositories · arXiv:2105.04553
-
Going deeper with Image Transformers 31 Mar 2021 · 21 repositories · arXiv:2103.17239Syntology official: no sample here; runs from other or unrecorded repositories · 9 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 2 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 11 harvested samples) · 2 pointer-only (licence)
-
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification 27 Mar 2021 · 15 repositories · arXiv:2103.14899Syntology official (archive's flag): 4 ran · 17 ran (of which 10 constructed an object rather than computing a result; 17 with no instrument failure: 0 honoured, 1 violated, 16 with no contract checked; 0 where Syntology's instrument failed) · 9 unverified (of 26 harvested samples)
-
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases 19 Mar 2021 · 9 repositories · arXiv:2103.10697
-
Transformer in Transformer 27 Feb 2021 · 12 repositories · arXiv:2103.00112Syntology official (archive's flag): 4 ran · 16 ran (of which 12 constructed an object rather than computing a result; 15 with no instrument failure: 1 honoured, 1 violated, 13 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified (of 24 harvested samples) · 5 pointer-only (licence)
-
Conditional Positional Encodings for Vision Transformers 22 Feb 2021 · 2 repositories · arXiv:2102.10882
-
Training data-efficient image transformers & distillation through attention 23 Dec 2020 · 40 repositories · arXiv:2012.12877Syntology community repositories only · 12 ran (of which 1 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 7 unverified (of 19 harvested samples) · 3 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 78 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections