Papers › SynCo: Synthetic Hard Negatives in Contrastive Learning for Better Unsupervised Visual...

SynCo: Synthetic Hard Negatives in Contrastive Learning for Better Unsupervised Visual Representations

3 Oct 2024arXiv:2410.02401archive 2025-07-28

Nikolaos Giakoumoglou, Tania Stathaki

Contrastive learning has become a dominant approach in self-supervised visual representation learning. Hard negatives - samples closely resembling the anchor - are key to enhancing learned representations' discriminative power. However, efficiently leveraging hard negatives remains challenging. We introduce SynCo (Synthetic Negatives in Contrastive learning), a novel approach that improves model performance by generating synthetic hard negatives on the representation space. Building on the MoCo framework, SynCo introduces six strategies for creating diverse synthetic hard negatives on-the-fly with minimal computational overhead. SynCo achieves faster training and better representation learning, reaching 67.9% top-1 accuracy on ImageNet ILSVRC-2012 linear evaluation after 200 pretraining epochs, surpassing MoCo's 67.5% using the same ResNet-50 encoder. It also transfers more effectively to detection tasks: on PASCAL VOC, it outperforms both the supervised baseline and MoCo with 82.5% AP; on COCO, it sets new benchmarks with 40.9% AP for bounding box detection and 35.5% AP for instance segmentation. Our synthetic hard negative generation approach significantly enhances visual representations learned through self-supervised contrastive learning. Code is available at https://github.com/giakoumoglou/synco.

Paper PDF Code

Code

giakoumoglou/synco officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Contrastive Learning Image Classification Image Segmentation Instance Segmentation Linear evaluation Object Detection Representation Learning Self-Supervised Image Classification Self-Supervised Learning Semantic Segmentation Semi-Supervised Image Classification Unsupervised Pre-training

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Image Segmentation	COCO val2017	SynCo (ResNet-50) 200ep	mask AP	35.4	#1 of 1	Archive leaderboard	report
Object Detection	COCO val2017	SynCo (ResNet-50) 200ep	Bounding Box AP	40.4	#1 of 1	Archive leaderboard	report
Object Detection	PASCAL VOC 2012 test	SynCo (ResNet-50) 200ep	Bounding Box AP	57.2	#1 of 1	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 800ep	Number of Params	24M	#101 of 144	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 800ep	Top 1 Accuracy	70.6%	#101 of 144	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 800ep	Top 5 Accuracy	89.8%	#101 of 144	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 200ep	Number of Params	24M	#108 of 144	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 200ep	Top 1 Accuracy	67.9%	#108 of 144	Archive leaderboard	report
Self-Supervised Image Classification	ImageNet	SynCo (ResNet-50) 200ep	Top 5 Accuracy	88	#108 of 144	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 1% labeled data	SynCo (ResNet-50) 800ep	Number of params	24M	#51 of 65	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 1% labeled data	SynCo (ResNet-50) 800ep	Top 1 Accuracy	50.8%	#51 of 65	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 1% labeled data	SynCo (ResNet-50) 800ep	Top 5 Accuracy	77.5%	#51 of 65	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 10% labeled data	SynCo (ResNet-50) 800ep	Number of params	24M	#44 of 75	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 10% labeled data	SynCo (ResNet-50) 800ep	Top 1 Accuracy	66.6%	#44 of 75	Archive leaderboard	report
Semi-Supervised Image Classification	ImageNet - 10% labeled data	SynCo (ResNet-50) 800ep	Top 5 Accuracy	88.0%	#44 of 75	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Batch Normalization Contrastive Learning Faster R-CNN InfoNCE Mask R-CNN MoCo

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections