{"url":"/method/moco","slug":"moco","name":"MoCo","full_name":"Momentum Contrast","full_name_withheld":false,"description_markdown":"**MoCo**, or **Momentum Contrast**, is a self-supervised learning algorithm with a contrastive loss. \r\n\r\nContrastive loss methods can be thought of as building dynamic dictionaries. The \"keys\" (tokens) in the dictionary are sampled from data (e.g., images or patches) and are represented by an encoder network. Unsupervised learning trains encoders to perform dictionary look-up: an encoded “query” should be similar to its matching key and dissimilar to others. Learning is formulated as minimizing a contrastive loss. \r\n\r\nMoCo can be viewed as a way to build large and consistent dictionaries for unsupervised learning with a contrastive loss. In MoCo, we maintain the dictionary as a queue of data samples: the encoded representations of the current mini-batch are enqueued, and the oldest are dequeued. The queue decouples the dictionary size from the mini-batch size, allowing it to be large. Moreover, as the dictionary keys come from the preceding several mini-batches, a slowly progressing key encoder, implemented as a momentum-based moving average of the query encoder, is proposed to maintain consistency.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1911.05722v3","title":"Momentum Contrast for Unsupervised Visual Representation Learning","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/facebookresearch/moco/blob/3631be074a0a14ab85c206631729fe035e54b525/moco/builder.py#L6","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Self-Supervised Learning","url":"/methods/category/self-supervised-learning","pwc_aliases":[]}],"n_papers_tagged":148,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/graph-supported-dynamic-algorithm","title":"Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization","date":"2025-05-22","arxiv_id":"2505.16471","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":null,"title":"Variational Self-Supervised Learning","date":"2025-04-06","arxiv_id":"2504.04318","n_code_links":0,"syntology":null},{"paper":null,"title":"Task-Specific Knowledge Distillation from the Vision Foundation Model for Enhanced Medical Image Segmentation","date":"2025-03-10","arxiv_id":"2503.06976","n_code_links":0,"syntology":null},{"paper":"/paper/dataset-ownership-verification-in-contrastive","title":"Dataset Ownership Verification in Contrastive Pre-trained Models","date":"2025-02-11","arxiv_id":"2502.07276","n_code_links":1,"syntology":null},{"paper":null,"title":"Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?","date":"2025-01-26","arxiv_id":"2501.15431","n_code_links":0,"syntology":null},{"paper":null,"title":"Language modulates vision: Evidence from neural networks and human brain-lesion models","date":"2025-01-23","arxiv_id":"2501.13628","n_code_links":0,"syntology":null},{"paper":"/paper/enhancing-contrastive-learning-inspired-by","title":"Enhancing Contrastive Learning Inspired by the Philosophy of \"The Blind Men and the Elephant\"","date":"2024-12-21","arxiv_id":"2412.16522","n_code_links":1,"syntology":null},{"paper":"/paper/scan-bootstrapping-contrastive-pre-training","title":"SCAN: Bootstrapping Contrastive Pre-training for Data Efficiency","date":"2024-11-14","arxiv_id":"2411.09126","n_code_links":1,"syntology":null},{"paper":null,"title":"Accelerating Augmentation Invariance Pretraining","date":"2024-10-27","arxiv_id":"2410.22364","n_code_links":0,"syntology":null},{"paper":null,"title":"SRA: A Novel Method to Improve Feature Embedding in Self-supervised Learning for Histopathological Images","date":"2024-10-23","arxiv_id":"2410.17514","n_code_links":0,"syntology":null},{"paper":"/paper/synco-synthetic-hard-negatives-in-contrastive","title":"SynCo: Synthetic Hard Negatives in Contrastive Learning for Better Unsupervised Visual Representations","date":"2024-10-03","arxiv_id":"2410.02401","n_code_links":1,"syntology":null},{"paper":"/paper/moner-motion-correction-in-undersampled","title":"Moner: Motion Correction in Undersampled Radial MRI with Unsupervised Neural Representation","date":"2024-09-25","arxiv_id":"2409.16921","n_code_links":1,"syntology":{"ran":4,"of":6,"unverified":2,"pointer_only":5}},{"paper":"/paper/upper-body-free-breathing-magnetic-resonance","title":"Upper-body free-breathing Magnetic Resonance Fingerprinting applied to the quantification of water T1 and fat fraction","date":"2024-09-24","arxiv_id":"2409.16200","n_code_links":1,"syntology":null},{"paper":"/paper/cross-model-cross-stream-learning-for-self","title":"Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition","date":"2024-09-23","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/drl-based-resource-allocation-for-motion-blur","title":"DRL-Based Resource Allocation for Motion Blur Resistant Federated Self-Supervised Learning in IoV","date":"2024-08-17","arxiv_id":"2408.09194","n_code_links":1,"syntology":null},{"paper":null,"title":"Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning","date":"2024-08-12","arxiv_id":"2408.05956","n_code_links":0,"syntology":null},{"paper":null,"title":"Contrastive Learning for Image Complexity Representation","date":"2024-08-06","arxiv_id":"2408.03230","n_code_links":0,"syntology":null},{"paper":null,"title":"Alignment Calibration: Machine Unlearning for Contrastive Learning under Auditing","date":"2024-06-05","arxiv_id":"2406.03603","n_code_links":0,"syntology":null},{"paper":null,"title":"WIDIn: Wording Image for Domain-Invariant Representation in Single-Source Domain Generalization","date":"2024-05-28","arxiv_id":"2405.18405","n_code_links":0,"syntology":null},{"paper":null,"title":"Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization","date":"2024-05-14","arxiv_id":"2405.08604","n_code_links":0,"syntology":null},{"paper":"/paper/additive-margin-in-contrastive-self","title":"Additive Margin in Contrastive Self-Supervised Frameworks to Learn Discriminative Speaker Representations","date":"2024-04-23","arxiv_id":"2404.14913","n_code_links":1,"syntology":null},{"paper":null,"title":"Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation","date":"2024-02-22","arxiv_id":"2402.14611","n_code_links":0,"syntology":null},{"paper":null,"title":"$f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning","date":"2024-02-15","arxiv_id":"2402.10150","n_code_links":0,"syntology":null},{"paper":"/paper/moco-a-learnable-meta-optimizer-for","title":"Moco: A Learnable Meta Optimizer for Combinatorial Optimization","date":"2024-02-07","arxiv_id":"2402.04915","n_code_links":1,"syntology":{"ran":9,"of":10,"unverified":1,"pointer_only":0}},{"paper":"/paper/diffclone-enhanced-behaviour-cloning-in","title":"DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning","date":"2024-01-17","arxiv_id":"2401.09243","n_code_links":1,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":0}},{"paper":null,"title":"A Beam-Segmenting Polar Format Algorithm Based on Double PCS for Video SAR Persistent Imaging","date":"2023-12-19","arxiv_id":"2401.10252","n_code_links":0,"syntology":null},{"paper":null,"title":"SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer","date":"2023-12-02","arxiv_id":"2312.01187","n_code_links":0,"syntology":null},{"paper":null,"title":"Maximum Entropy Model Correction in Reinforcement Learning","date":"2023-11-29","arxiv_id":"2311.17855","n_code_links":0,"syntology":null},{"paper":null,"title":"MoCo-Transfer: Investigating out-of-distribution contrastive learning for limited-data domains","date":"2023-11-15","arxiv_id":"2311.09401","n_code_links":0,"syntology":null},{"paper":null,"title":"SAMCLR: Contrastive pre-training on complex scenes using SAM for view sampling","date":"2023-10-23","arxiv_id":"2310.14736","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":62},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":61},{"task":"/task/representation-learning","name":"Representation Learning","papers":49},{"task":"/task/image-classification","name":"Image Classification","papers":20},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":16},{"task":"/task/object-detection","name":"Object Detection","papers":14},{"task":"/task/linear-evaluation","name":"Linear evaluation","papers":13},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":13},{"task":"/task/object-detection-1","name":"object-detection","papers":13},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":12},{"task":"/task/image-classification","name":"image-classification","papers":11},{"task":"/task/self-supervised-image-classification","name":"Self-Supervised Image Classification","papers":8},{"task":"/task/combinatorial-optimization","name":"Combinatorial Optimization","papers":7},{"task":"/task/action-recognition-in-videos","name":"Action Recognition","papers":6},{"task":"/task/classification-1","name":"Classification","papers":6},{"task":"/task/classification","name":"General Classification","papers":5},{"task":"/task/retrieval","name":"Retrieval","papers":5},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":4},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":4},{"task":"/task/language-modeling","name":"Language Modeling","papers":4}],"tasks_shown":20,"n_tasks":165,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":24},{"year":"2021","papers":45},{"year":"2022","papers":32},{"year":"2023","papers":21},{"year":"2024","papers":19},{"year":"2025","papers":6}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/moco"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}