Papers › All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reduction

All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reduction

16 Mar 2023ICCV 2023 1arXiv:2303.09417archive 2025-07-28

Imanol G. Estepa, Ignacio Sarasúa, Bhalaji Nagarajan, Petia Radeva

Nearest neighbour based methods have proved to be one of the most successful self-supervised learning (SSL) approaches due to their high generalization capabilities. However, their computational efficiency decreases when more than one neighbour is used. In this paper, we propose a novel contrastive SSL approach, which we call All4One, that reduces the distance between neighbour representations using ''centroids'' created through a self-attention mechanism. We use a Centroid Contrasting objective along with single Neighbour Contrasting and Feature Contrasting objectives. Centroids help in learning contextual information from multiple neighbours whereas the neighbour contrast enables learning representations directly from the neighbours and the feature contrast allows learning representations unique to the features. This combination enables All4One to outperform popular instance discrimination approaches by more than 1% on linear classification evaluation for popular benchmark datasets and obtains state-of-the-art (SoTA) results. Finally, we show that All4One is robust towards embedding dimensionalities and augmentations, surpassing NNCLR and Barlow Twins by more than 5% on low dimensionality and weak augmentation settings. The source code would be made available soon.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

ImaGonEs/all4one officialpytorchSyntology: not harvested report
vturrisi/solo-learn officialpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Improved Baselines with Momentum Contrastive Learning (arXiv:2003.04297; that paper's own run record: 28 ran (of which 0 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 1 violated, 20 with no contract checked; 7 where Syntology's instrument failed) · 15 unverified), Whitening for Self-Supervised Representation Learning (arXiv:2007.06346; that paper's own run record: 11 ran (of which 3 constructed an object rather than computing a result; 9 with no instrument failure: 1 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified) and ReSSL: Relational Self-Supervised Learning with Weak Augmentation (arXiv:2107.09282; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Computational EfficiencyContrastive LearningSelf-Supervised Image ClassificationSelf-Supervised Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

NNCLR

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections