{"url":"/method/supervised-contrastive-loss","slug":"supervised-contrastive-loss","name":"Supervised Contrastive Loss","full_name":"Supervised Contrastive Loss","full_name_withheld":false,"description_markdown":"**Supervised Contrastive Loss** is an alternative loss function to cross entropy that the authors argue can leverage label information more effectively. Clusters of points belonging to the same class are pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes.\r\n\r\n$$\r\n  \\mathcal{L}^{sup}=\\sum_{i=1}^{2N}\\mathcal{L}_i^{sup}\r\n  \\label{eqn:total_supervised_loss}\r\n$$\r\n\r\n$$\r\n  \\mathcal{L}\\_i^{sup}=\\frac{-1}{2N\\_{\\boldsymbol{\\tilde{y}}\\_i}-1}\\sum\\_{j=1}^{2N}\\mathbf{1}\\_{i\\neq j}\\cdot\\mathbf{1}\\_{\\boldsymbol{\\tilde{y}}\\_i=\\boldsymbol{\\tilde{y}}_j}\\cdot\\log{\\frac{\\exp{\\left(\\boldsymbol{z}\\_i\\cdot\\boldsymbol{z}\\_j/\\tau\\right)}}{\\sum\\_{k=1}^{2N}\\mathbf{1}\\_{i\\neq k}\\cdot\\exp{\\left(\\boldsymbol{z}\\_i\\cdot\\boldsymbol{z}\\_k/\\tau\\right)}}}\r\n$$\r\n\r\nwhere $N_{\\boldsymbol{\\tilde{y}}_i}$ is the total number of images in the minibatch that have the same label, $\\boldsymbol{\\tilde{y}}_i$, as the anchor, $i$. This loss has important properties well suited for supervised learning: (a) generalization to an arbitrary number of positives, (b) contrastive power increases with more negatives.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/2004.11362v5","title":"Supervised Contrastive Learning","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/HobbitLong/SupContrast/blob/8117ece43eea5a38fb154f587601ceeadbe416b3/losses.py#L11","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Loss Functions","url":"/methods/category/loss-functions","pwc_aliases":[]}],"n_papers_tagged":76,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning","date":"2025-06-04","arxiv_id":"2506.04411","n_code_links":0,"syntology":null},{"paper":null,"title":"MSA-UNet3+: Multi-Scale Attention UNet3+ with New Supervised Prototypical Contrastive Loss for Coronary DSA Image Segmentation","date":"2025-04-07","arxiv_id":"2504.05184","n_code_links":0,"syntology":null},{"paper":null,"title":"DFCon: Attention-Driven Supervised Contrastive Learning for Robust Deepfake Detection","date":"2025-01-28","arxiv_id":"2501.16704","n_code_links":0,"syntology":null},{"paper":null,"title":"An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise","date":"2025-01-02","arxiv_id":"2501.01130","n_code_links":0,"syntology":null},{"paper":null,"title":"Transformer-Based Contrastive Meta-Learning For Low-Resource Generalizable Activity Recognition","date":"2024-12-28","arxiv_id":"2412.20290","n_code_links":0,"syntology":null},{"paper":"/paper/multi-label-contrastive-learning-a","title":"Multi-Label Contrastive Learning : A Comprehensive Study","date":"2024-11-27","arxiv_id":"2412.00101","n_code_links":1,"syntology":null},{"paper":null,"title":"JEMA: A Joint Embedding Framework for Scalable Co-Learning with Multimodal Alignment","date":"2024-10-31","arxiv_id":"2410.23988","n_code_links":0,"syntology":null},{"paper":"/paper/fedccrl-federated-domain-generalization-with","title":"FedCCRL: Federated Domain Generalization with Cross-Client Representation Learning","date":"2024-10-15","arxiv_id":"2410.11267","n_code_links":1,"syntology":null},{"paper":null,"title":"A Unified Contrastive Loss for Self-Training","date":"2024-09-11","arxiv_id":"2409.07292","n_code_links":0,"syntology":null},{"paper":null,"title":"Out of spuriousity: Improving robustness to spurious correlations without group annotations","date":"2024-07-20","arxiv_id":"2407.14974","n_code_links":0,"syntology":null},{"paper":"/paper/etscl-an-evidence-theory-based-supervised","title":"ETSCL: An Evidence Theory-Based Supervised Contrastive Learning Framework for Multi-modal Glaucoma Grading","date":"2024-07-19","arxiv_id":"2407.14230","n_code_links":1,"syntology":null},{"paper":"/paper/exploring-multi-view-pixel-contrast-for","title":"Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization","date":"2024-06-19","arxiv_id":"2406.13565","n_code_links":1,"syntology":null},{"paper":null,"title":"Unleashing the Potential of Open-set Noisy Samples Against Label Noise for Medical Image Classification","date":"2024-06-18","arxiv_id":"2406.12293","n_code_links":0,"syntology":null},{"paper":null,"title":"Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Loss","date":"2024-06-10","arxiv_id":"2406.06848","n_code_links":0,"syntology":null},{"paper":"/paper/classification-of-breast-cancer","title":"Classification of Breast Cancer Histopathology Images using a Modified Supervised Contrastive Learning Method","date":"2024-05-06","arxiv_id":"2405.03642","n_code_links":1,"syntology":null},{"paper":"/paper/dacad-domain-adaptation-contrastive-learning","title":"DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series","date":"2024-04-17","arxiv_id":"2404.11269","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":0}},{"paper":null,"title":"Fair Graph Neural Network with Supervised Contrastive Regularization","date":"2024-04-09","arxiv_id":"2404.06090","n_code_links":0,"syntology":null},{"paper":"/paper/time-series-representation-learning-with","title":"Time Series Representation Learning with Supervised Contrastive Temporal Transformer","date":"2024-03-16","arxiv_id":"2403.10787","n_code_links":1,"syntology":null},{"paper":"/paper/neighborhood-enhanced-supervised-contrastive","title":"Neighborhood-Enhanced Supervised Contrastive Learning for Collaborative Filtering","date":"2024-02-18","arxiv_id":"2402.11523","n_code_links":1,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":0}},{"paper":null,"title":"Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning","date":"2023-12-26","arxiv_id":"2312.15869","n_code_links":0,"syntology":null},{"paper":null,"title":"Long-Tail Learning with Rebalanced Contrastive Loss","date":"2023-12-04","arxiv_id":"2312.01753","n_code_links":0,"syntology":null},{"paper":null,"title":"Contrastive Learning of View-Invariant Representations for Facial Expressions Recognition","date":"2023-11-12","arxiv_id":"2311.06852","n_code_links":0,"syntology":null},{"paper":"/paper/understanding-writing-style-in-social-media","title":"Understanding writing style in social media with a supervised contrastively pre-trained transformer","date":"2023-10-17","arxiv_id":"2310.11081","n_code_links":1,"syntology":null},{"paper":null,"title":"Semi-Supervised End-To-End Contrastive Learning For Time Series Classification","date":"2023-10-13","arxiv_id":"2310.08848","n_code_links":0,"syntology":null},{"paper":null,"title":"Continual Contrastive Spoken Language Understanding","date":"2023-10-04","arxiv_id":"2310.02699","n_code_links":0,"syntology":null},{"paper":null,"title":"SALSA: Semantically-Aware Latent Space Autoencoder","date":"2023-10-04","arxiv_id":"2310.02744","n_code_links":0,"syntology":null},{"paper":null,"title":"When hard negative sampling meets supervised contrastive learning","date":"2023-08-28","arxiv_id":"2308.14893","n_code_links":0,"syntology":null},{"paper":null,"title":"RAHNet: Retrieval Augmented Hybrid Network for Long-tailed Graph Classification","date":"2023-08-04","arxiv_id":"2308.02335","n_code_links":0,"syntology":null},{"paper":"/paper/contrastive-learning-for-api-aspect-analysis","title":"Contrastive Learning for API Aspect Analysis","date":"2023-07-31","arxiv_id":"2307.16878","n_code_links":2,"syntology":null},{"paper":null,"title":"Neural Memory Decoding with EEG Data and Representation Learning","date":"2023-07-25","arxiv_id":"2307.13181","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":49},{"task":"/task/representation-learning","name":"Representation Learning","papers":22},{"task":"/task/image-classification","name":"Image Classification","papers":11},{"task":"/task/classification-1","name":"Classification","papers":5},{"task":"/task/long-tail-learning","name":"Long-tail Learning","papers":5},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":5},{"task":"/task/image-classification","name":"image-classification","papers":5},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":4},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":4},{"task":"/task/time-series-1","name":"Time Series","papers":4},{"task":"/task/attribute","name":"Attribute","papers":3},{"task":"/task/class-incremental-learning","name":"Class Incremental Learning","papers":3},{"task":"/task/continual-learning","name":"Continual Learning","papers":3},{"task":"/task/diagnostic","name":"Diagnostic","papers":3},{"task":"/task/domain-generalization","name":"Domain Generalization","papers":3},{"task":"/task/face-swapping","name":"Face Swapping","papers":3},{"task":"/task/knowledge-distillation","name":"Knowledge Distillation","papers":3},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":3},{"task":"/task/change-detection","name":"Change Detection","papers":2},{"task":"/task/decoder","name":"Decoder","papers":2}],"tasks_shown":20,"n_tasks":132,"usage_by_year":[{"year":"2020","papers":2},{"year":"2021","papers":21},{"year":"2022","papers":16},{"year":"2023","papers":18},{"year":"2024","papers":15},{"year":"2025","papers":4}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/supervised-contrastive-loss"},"syntology_read_at":"2026-09-25T09:33:49+00:00"}