{"url":"/method/infonce","slug":"infonce","name":"InfoNCE","full_name":"InfoNCE","full_name_withheld":false,"description_markdown":"**InfoNCE**, where NCE stands for Noise-Contrastive Estimation, is a type of contrastive loss function used for [self-supervised learning](https://paperswithcode.com/methods/category/self-supervised-learning).\r\n\r\nGiven a set $X = ${$x\\_{1}, \\dots, x\\_{N}$} of $N$ random samples containing one positive sample from $p\\left(x\\_{t+k}|c\\_{t}\\right)$ and $N − 1$ negative samples from the 'proposal' distribution $p\\left(x\\_{t+k}\\right)$, we optimize:\r\n\r\n$$ \\mathcal{L}\\_{N} = - \\mathbb{E}\\_{X}\\left[\\log\\frac{f\\_{k}\\left(x\\_{t+k}, c\\_{t}\\right)}{\\sum\\_{x\\_{j}\\in{X}}f\\_{k}\\left(x\\_{j}, c\\_{t}\\right)}\\right] $$\r\n\r\nOptimizing this loss will result in $f\\_{k}\\left(x\\_{t+k}, c\\_{t}\\right)$ estimating the density ratio, which is:\r\n\r\n$$ f\\_{k}\\left(x\\_{t+k}, c\\_{t}\\right) \\propto \\frac{p\\left(x\\_{t+k}|c\\_{t}\\right)}{p\\left(x\\_{t+k}\\right)} $$","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1807.03748v2","title":"Representation Learning with Contrastive Predictive Coding","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/jefflai108/Contrastive-Predictive-Coding-PyTorch/blob/dfe687cf463668b16b0c2e205a166dbfbc9db227/src/model/model.py#L98","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Loss Functions","url":"/methods/category/loss-functions","pwc_aliases":[]}],"n_papers_tagged":418,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Generalizing Supervised Contrastive learning: A Projection Perspective","date":"2025-06-11","arxiv_id":"2506.09810","n_code_links":0,"syntology":null},{"paper":null,"title":"Probabilistic Variational Contrastive Learning","date":"2025-06-11","arxiv_id":"2506.10159","n_code_links":0,"syntology":null},{"paper":"/paper/integration-of-contrastive-predictive-coding","title":"Integration of Contrastive Predictive Coding and Spiking Neural Networks","date":"2025-06-10","arxiv_id":"2506.09194","n_code_links":1,"syntology":null},{"paper":"/paper/conventional-contrastive-learning-often-falls","title":"Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval with Cross-Encoder Listwise Distillation and Synthetic Data","date":"2025-05-25","arxiv_id":"2505.19274","n_code_links":1,"syntology":null},{"paper":"/paper/graph-supported-dynamic-algorithm","title":"Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization","date":"2025-05-22","arxiv_id":"2505.16471","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/rebalancing-contrastive-alignment-with","title":"Contrastive Alignment with Semantic Gap-Aware Corrections in Text-Video Retrieval","date":"2025-05-18","arxiv_id":"2505.12499","n_code_links":1,"syntology":null},{"paper":null,"title":"Adversarial Robustness for Unified Multi-Modal Encoders via Efficient Calibration","date":"2025-05-17","arxiv_id":"2505.11895","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-cross Feature based Spiking Neural Networks for Efficient Few-shot Learning","date":"2025-05-12","arxiv_id":"2505.07921","n_code_links":0,"syntology":null},{"paper":null,"title":"sEEG-based Encoding for Sentence Retrieval: A Contrastive Learning Approach to Brain-Language Alignment","date":"2025-04-20","arxiv_id":"2504.14468","n_code_links":0,"syntology":null},{"paper":null,"title":"Koopman-Based Event-Triggered Control from Data","date":"2025-04-19","arxiv_id":"2504.14334","n_code_links":0,"syntology":null},{"paper":null,"title":"Variational Self-Supervised Learning","date":"2025-04-06","arxiv_id":"2504.04318","n_code_links":0,"syntology":null},{"paper":null,"title":"Node Embeddings via Neighbor Embeddings","date":"2025-03-31","arxiv_id":"2503.23822","n_code_links":0,"syntology":null},{"paper":"/paper/beyond-contrastive-learning-synthetic-data","title":"Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance","date":"2025-03-29","arxiv_id":"2503.23239","n_code_links":1,"syntology":null},{"paper":"/paper/does-gcl-need-a-large-number-of-negative","title":"Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling","date":"2025-03-23","arxiv_id":"2503.17908","n_code_links":1,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":0}},{"paper":"/paper/weighted-graph-structure-learning-with","title":"Weighted Graph Structure Learning with Attention Denoising for Node Classification","date":"2025-03-15","arxiv_id":"2503.12157","n_code_links":1,"syntology":null},{"paper":"/paper/oasis-order-augmented-strategy-for-improved","title":"OASIS: Order-Augmented Strategy for Improved Code Search","date":"2025-03-11","arxiv_id":"2503.08161","n_code_links":0,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":1}},{"paper":null,"title":"Advancing Vietnamese Information Retrieval with Learning Objective and Benchmark","date":"2025-03-10","arxiv_id":"2503.07470","n_code_links":0,"syntology":null},{"paper":null,"title":"Task-Specific Knowledge Distillation from the Vision Foundation Model for Enhanced Medical Image Segmentation","date":"2025-03-10","arxiv_id":"2503.06976","n_code_links":0,"syntology":null},{"paper":null,"title":"Learning Transformer-based World Models with Contrastive Predictive Coding","date":"2025-03-06","arxiv_id":"2503.04416","n_code_links":0,"syntology":null},{"paper":null,"title":"LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning","date":"2025-03-04","arxiv_id":"2503.04812","n_code_links":0,"syntology":null},{"paper":"/paper/teaching-dense-retrieval-models-to-specialize","title":"Teaching Dense Retrieval Models to Specialize with Listwise Distillation and LLM Data Augmentation","date":"2025-02-27","arxiv_id":"2502.19712","n_code_links":1,"syntology":null},{"paper":null,"title":"Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence","date":"2025-02-24","arxiv_id":"2502.17028","n_code_links":0,"syntology":null},{"paper":"/paper/dataset-ownership-verification-in-contrastive","title":"Dataset Ownership Verification in Contrastive Pre-trained Models","date":"2025-02-11","arxiv_id":"2502.07276","n_code_links":1,"syntology":null},{"paper":null,"title":"Temperature-Free Loss Function for Contrastive Learning","date":"2025-01-29","arxiv_id":"2501.17683","n_code_links":0,"syntology":null},{"paper":null,"title":"CSPCL: Category Semantic Prior Contrastive Learning for Deformable DETR-Based Prohibited Item Detectors","date":"2025-01-28","arxiv_id":"2501.16665","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-supervised Benchmark Lottery on ImageNet: Do Marginal Improvements Translate to Improvements on Similar Datasets?","date":"2025-01-26","arxiv_id":"2501.15431","n_code_links":0,"syntology":null},{"paper":null,"title":"Contrastive Representation Learning Helps Cross-institutional Knowledge Transfer: A Study in Pediatric Ventilation Management","date":"2025-01-23","arxiv_id":"2501.13587","n_code_links":0,"syntology":null},{"paper":null,"title":"Language modulates vision: Evidence from neural networks and human brain-lesion models","date":"2025-01-23","arxiv_id":"2501.13628","n_code_links":0,"syntology":null},{"paper":null,"title":"An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise","date":"2025-01-02","arxiv_id":"2501.01130","n_code_links":0,"syntology":null},{"paper":null,"title":"Performance-Barrier Event-Triggered PDE Control of Traffic Flow","date":"2025-01-01","arxiv_id":"2501.00722","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":172},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":110},{"task":"/task/representation-learning","name":"Representation Learning","papers":104},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":28},{"task":"/task/retrieval","name":"Retrieval","papers":27},{"task":"/task/image-classification","name":"Image Classification","papers":26},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":25},{"task":"/task/image-classification","name":"image-classification","papers":18},{"task":"/task/object-detection","name":"Object Detection","papers":17},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":17},{"task":"/task/linear-evaluation","name":"Linear evaluation","papers":16},{"task":"/task/object-detection-1","name":"object-detection","papers":16},{"task":"/task/self-supervised-image-classification","name":"Self-Supervised Image Classification","papers":15},{"task":"/task/classification-1","name":"Classification","papers":13},{"task":"/task/classification","name":"General Classification","papers":11},{"task":"/task/language-modelling","name":"Language Modelling","papers":11},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":10},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":10},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":9},{"task":"/task/clustering","name":"Clustering","papers":9}],"tasks_shown":20,"n_tasks":343,"usage_by_year":[{"year":"2018","papers":2},{"year":"2019","papers":10},{"year":"2020","papers":52},{"year":"2021","papers":97},{"year":"2022","papers":89},{"year":"2023","papers":74},{"year":"2024","papers":64},{"year":"2025","papers":30}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/infonce"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}