Papers › Momentum Contrast for Unsupervised Visual Representation Learning

Momentum Contrast for Unsupervised Visual Representation Learning

13 Nov 2019CVPR 2020 6arXiv:1911.05722archive 2025-07-28

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick

We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1911.05722")

Code

Syntology Ran 26 of 42 code samples harvested from 24 repositories linked to this paper; 16 have no recorded run. Of those that ran: 3 ran · our draft was wrong; 2 ran · fixture could not drive it; 21 ran with no contract checked.

By repository: community (archive-listed): 42 samples from 24 repositories, 26 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

44 repositories listed; official and paper-mentioned ones first.

facebookresearch/moco officialmentioned in papermentioned on GitHubpytorch report
ppwwyyxx/moco.tensorflow officialmentioned on GitHubtf report
1170500804/MoCo mentioned on GitHubpytorch report
AidenDurrant/MoCo-Pytorch mentioned on GitHubpytorch report
AndrewTal/MoCo-Pytorch mentioned on GitHubpytorch report
CoinCheung/denseCL mentioned on GitHubpytorch report
Evgeneus/MoCo mentioned on GitHubpytorch report
HobbitLong/CMC mentioned on GitHubpytorchBSD-2-Clause report
KevinMusgrave/pytorch-metric-learning mentioned on GitHubpytorchMIT report
KevinMusgrave/pytorch_metric_learning mentioned on GitHubpytorch report
LeiLiu-byte/the-Essence-of-LT-Rocognition mentioned on GitHubpytorchnot reachable when probed 2026-09-17 — repositories for recent papers often appear after camera-ready report
PaperCodeReview/MoCo-TF mentioned on GitHubtfMIT report
Westlake-AI/openmixup mentioned on GitHubpytorch report
YyzHarry/imbalanced-semi-self mentioned on GitHubpytorch report
ali-nsua/DISCERN mentioned on GitHubpytorch report
alihassanijr/DISCERN mentioned on GitHubpytorch report
bl0/moco mentioned on GitHubpytorch report
davidsvy/hard-negative-mixing mentioned on GitHubpytorch report
ehsanik/muscleTorch mentioned on GitHubpytorch report
ernie1/pi-nas mentioned on GitHubpytorch report
facebookresearch/vissl mentioned on GitHubpytorch report
giakoumoglou/synco mentioned on GitHubpytorch report
joshr17/IFM mentioned on GitHubpytorch report
kikacaty/adv_guide mentioned on GitHubpytorch report
leftthomas/moco mentioned on GitHubpytorch report
lightly-ai/lightly mentioned on GitHubpytorch report
linwenye/light-moco mentioned on GitHubpytorch report
open-mmlab/mmselfsup mentioned on GitHubpytorchApache-2.0 report
smartdanny/MoCoV2_CIFAR10 mentioned on GitHubpytorch report
srinidhiPY/SSL_CR_Histo mentioned on GitHubpytorch report
szq0214/CMC_with_Image_Mixture mentioned on GitHubpytorch report
tranc012/SMILE-Rep mentioned on GitHubpytorchnot reachable when probed 2026-09-17 — repositories for recent papers often appear after camera-ready report
trannhiem/ma_ssrl_pytorch mentioned on GitHubpytorch report
xiangli13/MoCo mentioned on GitHubtf report
zhangyifei01/EIR mentioned on GitHubpytorch report
open-mmlab/mmdetection pytorchApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

42 samples harvested; 26 ran; 0 honoured the contract we drafted; 16 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

3ran · our draft was wrong
2ran · fixture could not drive it
21ran
16unverified

Licence: 16 of the 42 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 24 repositories linked to this paper, official or community; each sample names its own and says which. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

BaseMoCo HobbitLong/PyContrast/pycontrast/memory/mem_moco.py community (archive-listed) ran no licence file found · pointer only · 844eb6ac74b615e9 · report
CosineMomentumScheduler Pangoraw/GeoSSL/geossl/moco.py community (archive-listed) ran MIT (permissive) · bd4622df07c12c27 · report
MemoryMoCo ehsanik/muscleTorch/utils/contrastive_utils.py community (archive-listed) ran · metamorphic tier: deterministic no licence file found · pointer only · 587d6926bc87730f · report
MemoryMoCo bl0/moco/moco/NCE/Contrast.py community (archive-listed) ran no licence file found · pointer only · dfdbae86713f4677 · report
MemoryMoCo szq0214/CMC_with_Image_Mixture/NCE/NCEAverage.py community (archive-listed) ran no licence file found · pointer only · 1b18a48a2e94a2b4 · report
MoCo giakou4/pyssl/builders/moco.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · 82980b2ad9007485 · report
MoCo joshr17/IFM/moco-master/moco/builder.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · a92f8d3efcd0fa73 · report
MoCo amazon-research/exponential-moving-average-normalization/models/moco.py community (archive-listed) ran · metamorphic tier: deterministic no licence file found · pointer only · 60e7c290602e7d6a · report
MoCo 1170500804/MoCo/moco/builder.py community (archive-listed) ran licence not identified · pointer only · 1f9e6eca512e3008 · report
MoCo davidsvy/hard-negative-mixing/src/moco.py community (archive-listed) ran MIT (permissive) · d0fb06c245f652b9 · report
MoCo AndrewTal/MoCo-Pytorch/moco/builder.py community (archive-listed) ran no licence file found · pointer only · d568cb5387bc22f3 · report
MoCo kikacaty/adv_guide/moco/builder.py community (archive-listed) ran licence not identified · pointer only · ce8d71b82b20b145 · report
MoCo Pangoraw/GeoSSL/geossl/moco.py community (archive-listed) ran MIT (permissive) · 54ef0c846a171c93 · report
MoCo zhangyifei01/EIR/moco/builder.py community (archive-listed) ran Apache-2.0 (permissive) · 6e34a9f59a466d7f · report
MoCoProjectionHead lightly-ai/lightly/lightly/models/moco.py community (archive-listed) ran MIT (permissive) · f0d8899a1cadcb6a · report
Model leftthomas/moco/model.py community (archive-listed) ran fingerprinted no licence file found · pointer only · 2e32520281111228 · report
ModuleWithRecords KevinMusgrave/pytorch_metric_learning/src/pytorch_metric_learning/losses/cross_batch_memory.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · 5bf7745f1e480d55 · report
MomentumUpdater trannhiem/ma_ssrl_pytorch/MA_SSRL/utils/momentum.py community (archive-listed) ran licence not identified · pointer only · d57c8e5463db2477 · report
ProjectionHead lightly-ai/lightly/lightly/models/moco.py community (archive-listed) ran MIT (permissive) · d76bde76567cd6c0 · report
RGBMoCo HobbitLong/PyContrast/pycontrast/memory/mem_moco.py community (archive-listed) ran fingerprinted no licence file found · pointer only · 205d8773b717c4eb · report
build_hidden_head amazon-research/exponential-moving-average-normalization/models/moco.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 3ce0bcff10714b54 · report
conv3x3 HobbitLong/CMC/models/resnet.py community (archive-listed) ran · our draft was wrong BSD-2-Clause (permissive) · fac5364e2f53c6db · report
get_shuffle_ids HobbitLong/CMC/train_moco_ins.py community (archive-listed) ran · fixture could not drive it BSD-2-Clause (permissive) · e2e91e39d8368ced · report
infonce_loss giakou4/pyssl/builders/moco.py community (archive-listed) ran · fixture could not drive it fingerprinted MIT (permissive) · 5cd3d9e4fc246bdf · report
projection_MLP AidenDurrant/MoCo-Pytorch/src/model/moco.py community (archive-listed) ran · metamorphic tier: deterministic no licence file found · pointer only · dc6a714ff10ed3d9 · report
set_optimizer HobbitLong/CMC/LinearProbing.py community (archive-listed) ran · our draft was wrong BSD-2-Clause (permissive) · 3267a33f67aebf53 · report
BaseLossWrapper KevinMusgrave/pytorch_metric_learning/src/pytorch_metric_learning/losses/cross_batch_memory.py community (archive-listed) unverified MIT (permissive) · 3576e4fdf9ea3c2e · report
CrossBatchMemory KevinMusgrave/pytorch_metric_learning/src/pytorch_metric_learning/losses/cross_batch_memory.py community (archive-listed) unverified MIT (permissive) · f5b8b2ab5c47caa7 · report
MoCo linwenye/light-moco/moco/builder.py community (archive-listed) unverified licence not identified · pointer only · c605eb4989b100b9 · report
MoCo beresandras/contrastive-classification-keras/algorithms.py community (archive-listed) unverified MIT (permissive) · 1318e70df1366b7a · report
MoCo lightly-ai/lightly/lightly/models/moco.py community (archive-listed) unverified MIT (permissive) · 05ac8df2dc56ea0e · report
MoCo xiangli13/MoCo/MoCo.py community (archive-listed) unverified MIT (permissive) · 766dfd5385bfa150 · report
MoCo Evgeneus/MoCo/moco/builder.py community (archive-listed) unverified licence not identified · pointer only · 88df9e4db87f31e3 · report
MoCo_Model AidenDurrant/MoCo-Pytorch/src/model/moco.py community (archive-listed) unverified no licence file found · pointer only · a70f3f9f550eb5c4 · report
MomentumContrastiveModel beresandras/contrastive-classification-keras/algorithms.py community (archive-listed) unverified MIT (permissive) · 521294b5f41beba8 · report
SynCo giakoumoglou/synco/synco/builder.py community (archive-listed) unverified MIT (permissive) · 1fb0a31e12db1fbd · report
_MomentumEncoderMixin lightly-ai/lightly/lightly/models/moco.py community (archive-listed) unverified MIT (permissive) · 40e7ae5bffc7ea30 · report
_deactivate_requires_grad lightly-ai/lightly/lightly/models/moco.py community (archive-listed) unverified MIT (permissive) · 914a6024d62fcd9a · report
_do_momentum_update lightly-ai/lightly/lightly/models/moco.py community (archive-listed) unverified MIT (permissive) · a44f04114cc14792 · report
accuracy HobbitLong/CMC/eval_moco_ins.py community (archive-listed) unverified BSD-2-Clause (permissive) · 913d82af53065418 · report
resnet18 HobbitLong/CMC/models/resnet.py community (archive-listed) unverified BSD-2-Clause (permissive) · e56061351deca46f · report
resnet34 HobbitLong/CMC/models/resnet.py community (archive-listed) unverified BSD-2-Clause (permissive) · f3aafd1c32c0381c · report

Tasks

Contrastive LearningImage ClassificationRepresentation LearningSelf-Supervised Image Classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Contrastive Learning imagenet-1k ResNet50 ImageNet Top-1 Accuracy 60.6 #11 of 14 Archive leaderboard report
Image Classification OmniBenchmark MoCoV2 Average Top-1 Accuracy 34.8 #14 of 22 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 4x) Number of Params 375M #105 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 4x) Top 1 Accuracy 68.6% #105 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 2x) Number of Params 94M #112 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50 2x) Top 1 Accuracy 65.4% #112 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50) Number of Params 24M #123 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet MoCo (ResNet-50) Top 1 Accuracy 60.6% #123 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet (finetuned) MoCo (Resnet-50) Top 1 Accuracy 77.3% #62 of 65 Archive leaderboard report
Self-Supervised Image Classification ImageNet (finetuned) MoCo (Resnet-50) Top 1 Accuracy 77.0% #64 of 65 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAverage PoolingBatch NormalizationBottleneck Residual BlockColorJitterConvolutionExponential DecayFPNFaster R-CNNGlobal Average PoolingInfoNCEKaiming InitializationMask R-CNNMax PoolingMoCoRPNRandom GrayscaleRandom Horizontal FlipRandom Resized CropReLUResidual BlockResidual ConnectionRoIAlignRoIPoolSGD with MomentumSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections