Methods › General › Self-Supervised Learning › SimCLR
SimCLR
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
SimCLR is a framework for contrastive learning of visual representations. It learns representations by maximizing agreement between differently augmented views of the same data example via a contrastive loss in the latent space. It consists of:
-
A stochastic data augmentation module that transforms any given data example randomly resulting in two correlated views of the same example, denoted 𝐱̃ᵢ and 𝐱̃ⱼ, which is considered a positive pair. SimCLR sequentially applies three simple augmentations: random cropping followed by resize back to the original size, random color distortions, and random Gaussian blur. The authors find random crop and color distortion is crucial to achieve good performance.
-
A neural network base encoder f(·) that extracts representation vectors from augmented data examples. The framework allows various choices of the network architecture without any constraints. The authors opt for simplicity and adopt ResNet to obtain hᵢ = f(𝐱̃ᵢ) = ResNet(𝐱̃ᵢ) where hᵢ ∈ℝᵈ is the output after the average pooling layer.
-
A small neural network projection head g(·) that maps representations to the space where contrastive loss is applied. Authors use a MLP with one hidden layer to obtain zᵢ = g(hᵢ) = W⁽²⁾σ(W⁽¹⁾hᵢ) where σ is a ReLU nonlinearity. The authors find it beneficial to define the contrastive loss on zᵢ’s rather than hᵢ’s.
-
A contrastive loss function defined for a contrastive prediction task. Given a set {𝐱̃ₖ} including a positive pair of examples 𝐱̃ᵢ and 𝐱̃ⱼ , the contrastive prediction task aims to identify 𝐱̃ⱼ in {𝐱̃ₖ}_(k≠i) for a given 𝐱̃ᵢ.
A minibatch of N examples is randomly sampled and the contrastive prediction task is defined on pairs of augmented examples derived from the minibatch, resulting in 2N data points. Negative examples are not sampled explicitly. Instead, given a positive pair, the other 2(N − 1) augmented examples within a minibatch are treated as negative examples. A NT-Xent (the normalized temperature-scaled cross entropy loss) loss function is used (see components).
Papers archive 2025-07-28
30 shown of 240, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Probabilistic Variational Contrastive Learning 11 Jun 2025 · 0 repositories · arXiv:2506.10159
-
scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data 10 Jun 2025 · 1 repository · arXiv:2506.10031
-
Circumventing Backdoor Space via Weight Symmetry 9 Jun 2025 · 1 repository · arXiv:2506.07467
-
SSPS: Self-Supervised Positive Sampling for Robust Self-Supervised Speaker Verification 20 May 2025 · 1 repository · arXiv:2505.14561
-
Representation Learning via Non-Contrastive Mutual Information 23 Apr 2025 · 0 repositories · arXiv:2504.16667
-
Impact of Language Guidance: A Reproducibility Study 10 Apr 2025 · 0 repositories · arXiv:2504.08140
-
Unsupervised Detection of Fraudulent Transactions in E-commerce Using Contrastive Learning 24 Mar 2025 · 0 repositories · arXiv:2503.18841
-
A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics 21 Mar 2025 · 0 repositories · arXiv:2503.17538
-
SinSim: Sinkhorn-Regularized SimCLR 13 Feb 2025 · 0 repositories · arXiv:2502.10478
-
Dataset Ownership Verification in Contrastive Pre-trained Models 11 Feb 2025 · 1 repository · arXiv:2502.07276
-
Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling 29 Jan 2025 · 1 repository · arXiv:2501.17772
-
Enhancing Contrastive Learning Inspired by the Philosophy of "The Blind Men and the Elephant" 21 Dec 2024 · 1 repository · arXiv:2412.16522
-
Self-Supervised Radiograph Anatomical Region Classification -- How Clean Is Your Real-World Data? 20 Dec 2024 · 0 repositories · arXiv:2412.15967
-
Maximising Histopathology Segmentation using Minimal Labels via Self-Supervision 19 Dec 2024 · 0 repositories · arXiv:2412.15389
-
AmCLR: Unified Augmented Learning for Cross-Modal Representations 10 Dec 2024 · 1 repository · arXiv:2412.07979
-
Mitigating Instance-Dependent Label Noise: Integrating Self-Supervised Pretraining with Pseudo-Label Refinement 6 Dec 2024 · 0 repositories · arXiv:2412.04898
-
Tight PAC-Bayesian Risk Certificates for Contrastive Learning 4 Dec 2024 · 1 repository · arXiv:2412.03486
-
Explorations in Self-Supervised Learning: Dataset Composition Testing for Object Classification 1 Dec 2024 · 0 repositories · arXiv:2412.00770
-
Breccia and basalt classification of thin sections of Apollo rocks with deep learning 28 Oct 2024 · 1 repository · arXiv:2410.21024
-
Accelerating Augmentation Invariance Pretraining 27 Oct 2024 · 0 repositories · arXiv:2410.22364
-
Rethinking Positive Pairs in Contrastive Learning 23 Oct 2024 · 0 repositories · arXiv:2410.18200
-
SigCLR: Sigmoid Contrastive Learning of Visual Representations 22 Oct 2024 · 0 repositories · arXiv:2410.17427
-
Self-Supervised Anomaly Detection in the Wild: Favor Joint Embeddings Methods 5 Oct 2024 · 0 repositories · arXiv:2410.04289
-
BiSSL: Enhancing the Alignment Between Self-Supervised Pretraining and Downstream Fine-Tuning via Bilevel Optimization 3 Oct 2024 · 0 repositories · arXiv:2410.02387
-
Understanding the Benefits of SimCLR Pre-Training in Two-Layer Convolutional Neural Networks 27 Sep 2024 · 0 repositories · arXiv:2409.18685
-
Self-supervised Pretraining for Cardiovascular Magnetic Resonance Cine Segmentation 26 Sep 2024 · 1 repository · arXiv:2409.18100
-
Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition 23 Sep 2024 · 1 repository
-
Contrastive Learning for Character Detection in Ancient Greek Papyri 16 Sep 2024 · 0 repositories · arXiv:2409.10156
-
Label-free Monitoring of Self-Supervised Learning Progress 10 Sep 2024 · 0 repositories · arXiv:2409.06612
-
Learning Tree-Structured Composition of Data Augmentation 26 Aug 2024 · 1 repository · arXiv:2408.14381
Tasks archive 2025-07-28
20 shown of 202 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections