Papers › Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning
Julien Denize, Jaonary Rabarisoa, Astrid Orcesi, Romain Hérault
Contrastive representation learning has proven to be an effective self-supervised learning method for images and videos. Most successful approaches are based on Noise Contrastive Estimation (NCE) and use different views of an instance as positives that should be contrasted with other instances, called negatives, that are considered as noise. However, several instances in a dataset are drawn from the same distribution and share underlying semantic information. A good data representation should contain relations between the instances, or semantic similarity and dissimilarity, that contrastive learning harms by considering all negatives as noise. To circumvent this issue, we propose a novel formulation of contrastive learning using semantic similarity between instances called Similarity Contrastive Estimation (SCE). Our training objective is a soft contrastive one that brings the positives closer and estimates a continuous distribution to push or pull negative instances based on their learned similarities. We validate empirically our approach on both image and video representation learning. We show that SCE performs competitively with the state of the art on the ImageNet linear evaluation protocol for fewer pretraining epochs and that it generalizes to several downstream image tasks. We also show that SCE reaches state-of-the-art results for pretraining video representation and that the learned representation can generalize to video downstream tasks.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Self-Supervised Action Recognition | HMDB51 | SCE (R3D-50) | Frozen | false | #4 of 48 | Archive leaderboard | report |
| Self-Supervised Action Recognition | HMDB51 | SCE (R3D-50) | Pre-Training Dataset | Kinetics400 | #4 of 48 | Archive leaderboard | report |
| Self-Supervised Action Recognition | HMDB51 | SCE (R3D-50) | Top-1 Accuracy | 74.7 | #4 of 48 | Archive leaderboard | report |
| Self-Supervised Action Recognition | UCF101 | SCE (R3D-50) | 3-fold Accuracy | 95.3 | #7 of 53 | Archive leaderboard | report |
| Self-Supervised Action Recognition | UCF101 | SCE (R3D-50) | Frozen | false | #7 of 53 | Archive leaderboard | report |
| Self-Supervised Action Recognition | UCF101 | SCE (R3D-50) | Pre-Training Dataset | Kinetics400 | #7 of 53 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections