{"url":"/method/moco-v2","slug":"moco-v2","name":"MoCo v2","full_name":"MoCo v2","full_name_withheld":false,"description_markdown":"**MoCo v2** is an improved version of the [Momentum Contrast](https://paperswithcode.com/method/moco) self-supervised learning algorithm. Motivated by the findings presented in the [SimCLR](https://paperswithcode.com/method/simclr) paper, authors:\r\n\r\n- Replace the 1-layer fully connected layer with a 2-layer MLP head with [ReLU](https://paperswithcode.com/method/relu) for the unsupervised training stage.\r\n- Include blur augmentation.\r\n- Use cosine learning rate schedule.\r\n\r\nThese modifications enable MoCo to outperform the state-of-the-art SimCLR with a smaller batch size and fewer epochs.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Improved Baselines with Momentum Contrastive Learning","paper":"/paper/improved-baselines-with-momentum-contrastive","first_author":"Xinlei Chen","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/improved-baselines-with-momentum-contrastive"},"source":{"url":"https://arxiv.org/abs/2003.04297v1","title":"Improved Baselines with Momentum Contrastive Learning","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/facebookresearch/moco","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Semi-Supervised Learning Methods","url":"/methods/category/semi-supervised-learning-methods","pwc_aliases":[]},{"area":"General","area_id":"general","collection":"Self-Supervised Learning","url":"/methods/category/self-supervised-learning","pwc_aliases":[]}],"n_papers_tagged":30,"archive_num_papers":30,"papers_newest_first":[{"paper":"/paper/enhancing-contrastive-learning-inspired-by","title":"Enhancing Contrastive Learning Inspired by the Philosophy of \"The Blind Men and the Elephant\"","date":"2024-12-21","arxiv_id":"2412.16522","n_code_links":1,"syntology":null},{"paper":null,"title":"Contrastive Learning for Image Complexity Representation","date":"2024-08-06","arxiv_id":"2408.03230","n_code_links":0,"syntology":null},{"paper":null,"title":"Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation","date":"2024-02-22","arxiv_id":"2402.14611","n_code_links":0,"syntology":null},{"paper":null,"title":"SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer","date":"2023-12-02","arxiv_id":"2312.01187","n_code_links":0,"syntology":null},{"paper":"/paper/kernel-ssl-kernel-kl-divergence-for-self","title":"Matrix Information Theory for Self-Supervised Learning","date":"2023-05-27","arxiv_id":"2305.17326","n_code_links":3,"syntology":{"ran":5,"of":5,"unverified":0,"pointer_only":5}},{"paper":"/paper/mobyv2al-self-supervised-active-learning-for","title":"MoBYv2AL: Self-supervised Active Learning for Image Classification","date":"2023-01-04","arxiv_id":"2301.01531","n_code_links":1,"syntology":null},{"paper":"/paper/revisiting-the-critical-factors-of","title":"Revisiting the Critical Factors of Augmentation-Invariant Representation Learning","date":"2022-07-30","arxiv_id":"2208.00275","n_code_links":1,"syntology":{"ran":2,"of":5,"unverified":3,"pointer_only":0}},{"paper":"/paper/dual-temperature-helps-contrastive-learning","title":"Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo","date":"2022-03-30","arxiv_id":"2203.17248","n_code_links":2,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/data-domain-aware-and-task-aware-pre-training","title":"DATA: Domain-Aware and Task-Aware Self-supervised Learning","date":"2022-03-17","arxiv_id":"2203.09041","n_code_links":1,"syntology":{"ran":0,"of":2,"unverified":2,"pointer_only":0}},{"paper":null,"title":"InsCon:Instance Consistency Feature Representation via Self-Supervised Learning","date":"2022-03-15","arxiv_id":"2203.07688","n_code_links":0,"syntology":null},{"paper":null,"title":"Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features","date":"2022-03-03","arxiv_id":"2203.01881","n_code_links":0,"syntology":null},{"paper":"/paper/energy-based-contrastive-learning-of-visual","title":"Energy-Based Contrastive Learning of Visual Representations","date":"2022-02-10","arxiv_id":"2202.04933","n_code_links":1,"syntology":{"ran":12,"of":14,"unverified":2,"pointer_only":14}},{"paper":"/paper/contrastive-object-level-pre-training-with","title":"Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning","date":"2021-11-26","arxiv_id":"2111.13651","n_code_links":1,"syntology":null},{"paper":"/paper/regioncl-can-simple-region-swapping","title":"RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?","date":"2021-11-24","arxiv_id":"2111.12309","n_code_links":2,"syntology":null},{"paper":"/paper/8-bit-optimizers-via-block-wise-quantization","title":"8-bit Optimizers via Block-wise Quantization","date":"2021-10-06","arxiv_id":"2110.02861","n_code_links":3,"syntology":null},{"paper":null,"title":"Piecing and Chipping: An effective solution for the information-erasing view generation in Self-supervised Learning","date":"2021-09-29","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Self-Supervised Visual Representations Learning by Contrastive Mask Prediction","date":"2021-08-18","arxiv_id":"2108.07954","n_code_links":0,"syntology":null},{"paper":"/paper/self-supervised-learning-with-swin","title":"Self-Supervised Learning with Swin Transformers","date":"2021-05-10","arxiv_id":"2105.04553","n_code_links":6,"syntology":null},{"paper":"/paper/jigsaw-clustering-for-unsupervised-visual","title":"Jigsaw Clustering for Unsupervised Visual Representation Learning","date":"2021-04-01","arxiv_id":"2104.00323","n_code_links":1,"syntology":null},{"paper":null,"title":"Deeply Unsupervised Patch Re-Identification for Pre-training Object Detectors","date":"2021-03-08","arxiv_id":"2103.04814","n_code_links":0,"syntology":null},{"paper":"/paper/seed-self-supervised-distillation-for-visual-1","title":"SEED: Self-supervised Distillation For Visual Representation","date":"2021-01-12","arxiv_id":"2101.04731","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/cem500k-a-large-scale-heterogeneous-unlabeled","title":"CEM500K – A large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning","date":"2020-12-11","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology","date":"2020-12-07","arxiv_id":"2012.03583","n_code_links":0,"syntology":null},{"paper":"/paper/how-well-do-self-supervised-models-transfer","title":"How Well Do Self-Supervised Models Transfer?","date":"2020-11-26","arxiv_id":"2011.13377","n_code_links":1,"syntology":{"ran":4,"of":5,"unverified":1,"pointer_only":0}},{"paper":null,"title":"Are all negatives created equal in contrastive instance discrimination?","date":"2020-10-13","arxiv_id":"2010.06682","n_code_links":0,"syntology":null},{"paper":"/paper/eqco-equivalent-rules-for-self-supervised-1","title":"EqCo: Equivalent Rules for Self-supervised Contrastive Learning","date":"2020-10-05","arxiv_id":"2010.01929","n_code_links":1,"syntology":null},{"paper":"/paper/unsupervised-feature-learning-by-cross-level","title":"Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination","date":"2020-08-09","arxiv_id":"2008.03813","n_code_links":2,"syntology":{"ran":2,"of":8,"unverified":6,"pointer_only":0}},{"paper":"/paper/parametric-instance-classification-for","title":"Parametric Instance Classification for Unsupervised Visual Feature Learning","date":"2020-06-25","arxiv_id":"2006.14618","n_code_links":1,"syntology":null},{"paper":"/paper/learning-to-classify-images-without-labels","title":"SCAN: Learning to Classify Images without Labels","date":"2020-05-25","arxiv_id":"2005.12320","n_code_links":2,"syntology":null},{"paper":"/paper/improved-baselines-with-momentum-contrastive","title":"Improved Baselines with Momentum Contrastive Learning","date":"2020-03-09","arxiv_id":"2003.04297","n_code_links":36,"syntology":{"ran":8,"of":43,"unverified":35,"pointer_only":10}}],"papers_shown":30,"tasks":[{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":16},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":13},{"task":"/task/representation-learning","name":"Representation Learning","papers":9},{"task":"/task/image-classification","name":"Image Classification","papers":8},{"task":"/task/linear-evaluation","name":"Linear evaluation","papers":6},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":6},{"task":"/task/object-detection","name":"Object Detection","papers":5},{"task":"/task/object-detection-1","name":"object-detection","papers":5},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":4},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":4},{"task":"/task/classification-1","name":"Classification","papers":3},{"task":"/task/clustering","name":"Clustering","papers":2},{"task":"/task/classification","name":"General Classification","papers":2},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/language-modelling","name":"Language Modelling","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/self-supervised-image-classification","name":"Self-Supervised Image Classification","papers":2},{"task":"/task/semi-supervised-image-classification","name":"Semi-Supervised Image Classification","papers":2}],"tasks_shown":20,"n_tasks":49,"usage_by_year":[{"year":"2020","papers":9},{"year":"2021","papers":9},{"year":"2022","papers":6},{"year":"2023","papers":3},{"year":"2024","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/moco-v2"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}