</> Code with Papers by Syntology

Browse State-of-the-Art › Self-Supervised Image Classification

Self-Supervised Image Classification

97 papers with code · 3 benchmarks · 3 datasets archive 2025-07-28

AdversarialAudioComputer CodeComputer VisionMedicalMethodologyMiscellaneousMusicNatural Language ProcessingReasoningSpeech

This is the task of image classification using representations learnt with self-supervised learning. Self-supervised methods generally involve a pretext task that is solved to learn a good representation and a loss function to learn with. One example of a loss function is an autoencoder based loss where the goal is reconstruction of an image pixel-by-pixel. A more popular recent example is a contrastive loss, which measure the similarity of sample pairs in a representation space, and where there can be a varying target instead of a fixed target to reconstruct (as in the case of autoencoders).

A common evaluation protocol is to train a linear classifier on top of (frozen) representations learnt by self-supervised methods. The leaderboards for the linear evaluation protocol can be found below. In practice, it is more common to fine-tune features on a downstream task. An alternative evaluation protocol therefore uses semi-supervised learning and finetunes on a % of the labels. The leaderboards for the finetuning protocol can be accessed here.

You may want to read some blog posts before reading the papers and checking the leaderboards:

There is also Yann LeCun's talk at AAAI-20 which you can watch here (35:00+).

( Image credit: A Simple Framework for Contrastive Learning of Visual Representations )

Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.

Benchmarks archive 2025-07-28

3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

DatasetBest model (first row in archive order)PaperCodeSyntologyCompare
ImageNet (144 rows) DINOv2+reg (ViT-g/14) Vision Transformers Need Registers code Syntology ran 4 of 20 samples · 16 unverified Compare
ImageNet (finetuned) (65 rows) DINOv2 (ViT-g/14, 448) DINOv2: Learning Robust Visual Features without Supervision code Syntology ran 21 of 46 samples · 25 unverified Compare
Chest X-ray images (1 row) DINO-CXR DINO-CXR: A self supervised method based on vision transformer for... — — Compare

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

ImageNetChest X-ray imagesBIOSCAN-5M

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Image Classification

Most implemented papers archive 2025-07-28

30 shown of 97 papers with code (110 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

  • A Simple Framework for Contrastive Learning of Visual Representations
    13 Feb 2020 96 repositories listed Syntology ran 79 of 137 samples · 58 unverified · 52 pointer-only (licence)
    This paper presents SimCLR: a simple framework for contrastive learning of visual representations.
  • Masked Autoencoders Are Scalable Vision Learners
    11 Nov 2021 58 repositories listed Syntology ran 71 of 137 samples · 66 unverified · 73 pointer-only (licence)
    Our MAE approach is simple: we mask random patches of the input image and reconstruct the missing pixels.
  • Momentum Contrast for Unsupervised Visual Representation Learning
    13 Nov 2019 44 repositories listed Syntology ran 26 of 42 samples · 16 unverified · 16 pointer-only (licence)
    This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning.
  • Colorful Image Colorization
    28 Mar 2016 39 repositories listed Syntology ran 32 of 73 samples · 41 unverified · 41 pointer-only (licence)
    We embrace the underlying uncertainty of the problem by posing it as a classification task and use class-rebalancing at training time to increase the diversity of colors in the result.
  • Improved Baselines with Momentum Contrastive Learning
    9 Mar 2020 36 repositories listed Syntology ran 8 of 43 samples · 35 unverified · 10 pointer-only (licence)
    Contrastive unsupervised learning has recently shown encouraging progress, e.
  • Emerging Properties in Self-Supervised Vision Transformers
    29 Apr 2021 32 repositories listed Syntology ran 5 of 20 samples · 15 unverified · 2 pointer-only (licence)
    In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets).
  • Bootstrap your own latent: A new approach to self-supervised Learning
    13 Jun 2020 31 repositories listed Syntology ran 62 of 79 samples · 17 unverified · 46 pointer-only (licence)
    From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view.
  • Representation Learning with Contrastive Predictive Coding
    10 Jul 2018 28 repositories listed Syntology ran 29 of 45 samples · 16 unverified · 22 pointer-only (licence)
    The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models.
  • DINOv2: Learning Robust Visual Features without Supervision
    14 Apr 2023 26 repositories listed Syntology ran 21 of 46 samples · 25 unverified · 12 pointer-only (licence)
    The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision.
  • Exploring Simple Siamese Representation Learning
    20 Nov 2020 26 repositories listed Syntology ran 34 of 39 samples · 5 unverified · 22 pointer-only (licence)
    Our experiments show that collapsing solutions do exist for the loss and structure, but a stop-gradient operation plays an essential role in preventing collapsing.
  • Barlow Twins: Self-Supervised Learning via Redundancy Reduction
    4 Mar 2021 24 repositories listed Syntology ran 21 of 26 samples · 5 unverified · 10 pointer-only (licence)
    This causes the embedding vectors of distorted versions of a sample to be similar, while minimizing the redundancy between the components of these vectors.
  • Unsupervised Representation Learning by Predicting Image Rotations
    21 Mar 2018 20 repositories listed Syntology ran 14 of 22 samples · 8 unverified · 18 pointer-only (licence)
    However, in order to successfully learn those features, they usually require massive amounts of manually labeled data, which is both expensive and impractical to scale.
  • ResMLP: Feedforward networks for image classification with data-efficient training
    7 May 2021 19 repositories listed Syntology ran 2 of 7 samples · 5 unverified
    We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification.
  • Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
    17 Jun 2020 18 repositories listed Syntology ran 13 of 17 samples · 4 unverified · 6 pointer-only (licence)
    In addition, we also propose a new data augmentation strategy, multi-crop, that uses a mix of views with different resolutions in place of two full-resolution views, without increasing the memory or compute requirements…
  • BEiT: BERT Pre-Training of Image Transformers
    15 Jun 2021 14 repositories listed Syntology ran 6 of 11 samples · 5 unverified
    We first "tokenize" the original image into visual tokens.
  • XCiT: Cross-Covariance Image Transformers
    17 Jun 2021 12 repositories listed Syntology ran 3 of 14 samples · 11 unverified · 3 pointer-only (licence)
    We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries.
  • An Empirical Study of Training Self-Supervised Vision Transformers
    5 Apr 2021 9 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
    In this work, we go back to basics and investigate the effects of several fundamental components for training self-supervised ViT.
  • Big Self-Supervised Models are Strong Semi-Supervised Learners
    17 Jun 2020 9 repositories listed Syntology ran 0 of 6 samples · 6 unverified
    The proposed semi-supervised learning algorithm can be summarized in three steps: unsupervised pretraining of a big ResNet model using SimCLRv2, supervised fine-tuning on a few labeled examples, and distillation with…
  • Deep Clustering for Unsupervised Learning of Visual Features
    15 Jul 2018 9 repositories listed Syntology ran 5 of 7 samples · 2 unverified · 4 pointer-only (licence)
    In this work, we present DeepCluster, a clustering method that jointly learns the parameters of a neural network and the cluster assignments of the resulting features.
  • Contrastive Multiview Coding
    13 Jun 2019 8 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 1 pointer-only (licence)
    We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the…
  • SimMIM: A Simple Framework for Masked Image Modeling
    18 Nov 2021 7 repositories listed Syntology ran 9 of 14 samples · 5 unverified
    We also leverage this approach to facilitate the training of a 3B model (SwinV2-G), that by 40× less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks.
  • Self-Supervised Learning of Pretext-Invariant Representations
    4 Dec 2019 7 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
    The goal of self-supervised learning from images is to construct image representations that are semantically meaningful via pretext tasks that do not require semantic annotations for a large training set of images.
  • Vision Transformers Need Registers
    28 Sep 2023 6 repositories listed Syntology ran 4 of 20 samples · 16 unverified · 2 pointer-only (licence)
    Transformers have recently emerged as a powerful tool for learning visual representations.
  • EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
    14 Nov 2022 6 repositories listed Syntology ran 1 of 3 samples · 2 unverified
    We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data.
  • Context Autoencoder for Self-Supervised Representation Learning
    7 Feb 2022 6 repositories listed
    The pretraining tasks include two tasks: masked representation prediction - predict the representations for the masked patches, and masked patch reconstruction - reconstruct the masked patches.
  • Masked Feature Prediction for Self-Supervised Visual Pre-Training
    16 Dec 2021 6 repositories listed
    We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models.
  • VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
    11 May 2021 6 repositories listed Syntology ran 11 of 17 samples · 6 unverified · 1 pointer-only (licence)
    Recent self-supervised methods for image representation learning are based on maximizing the agreement between embedding vectors from different views of the same image.
  • Self-Supervised Learning with Swin Transformers
    10 May 2021 6 repositories listed
    We are witnessing a modeling shift from CNN to Transformers in computer vision.
  • Revisiting Self-Supervised Visual Representation Learning
    25 Jan 2019 6 repositories listed Syntology ran 0 of 15 samples · 15 unverified · 7 pointer-only (licence)
    Unsupervised visual representation learning remains a largely unsolved problem in computer vision research.
  • Self-labelling via simultaneous clustering and representation learning
    13 Nov 2019 5 repositories listed Syntology ran 7 of 18 samples · 11 unverified · 3 pointer-only (licence)
    Combining clustering and representation learning is one of the most promising approaches for unsupervised learning of deep neural networks.

Syntology lines on 27 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections