{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prototypical-contrastive-learning-of","title":"Prototypical Contrastive Learning of Unsupervised Representations","arxiv_id":"2005.04966","date":"2020-05-11","proceeding":"ICLR 2021 1","authors":["Junnan Li","Pan Zhou","Caiming Xiong","Steven C. H. Hoi"],"abstract":"This paper presents Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that addresses the fundamental limitations of instance-wise contrastive learning. PCL not only learns low-level features for the task of instance discrimination, but more importantly, it implicitly encodes semantic structures of the data into the learned embedding space. Specifically, we introduce prototypes as latent variables to help find the maximum-likelihood estimation of the network parameters in an Expectation-Maximization framework. We iteratively perform E-step as finding the distribution of prototypes via clustering and M-step as optimizing the network via contrastive learning. We propose ProtoNCE loss, a generalized version of the InfoNCE loss for contrastive learning, which encourages representations to be closer to their assigned prototypes. PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks with substantial improvement in low-resource transfer learning. Code and pretrained models are available at https://github.com/salesforce/PCL.","url_abs":"https://arxiv.org/abs/2005.04966v5","url_pdf":"https://arxiv.org/pdf/2005.04966v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prototypical-contrastive-learning-of","repo_url":"https://github.com/salesforce/PCL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"prototypical-contrastive-learning-of","repo_url":"https://github.com/salesforce/MoPro","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-image-classification","task_name":"Self-Supervised Image Classification"},{"task_slug":"semi-supervised-image-classification","task_name":"Semi-Supervised Image Classification"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"infonce","method_name":"InfoNCE"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/contrastive-learning-on-imagenet-1k","task":"Contrastive Learning","dataset":"imagenet-1k","model":"ResNet50 (v2)","rank_in_archive_order":5,"of":14,"metrics":{"ImageNet Top-1 Accuracy":"67.6"},"uses_additional_data":false},{"leaderboard":"/sota/contrastive-learning-on-imagenet-1k","task":"Contrastive Learning","dataset":"imagenet-1k","model":"ResNet50","rank_in_archive_order":8,"of":14,"metrics":{"ImageNet Top-1 Accuracy":"61.5"},"uses_additional_data":false},{"leaderboard":"/sota/self-supervised-image-classification-on","task":"Self-Supervised Image Classification","dataset":"ImageNet","model":"PCL (ResNet-50)","rank_in_archive_order":111,"of":144,"metrics":{"Number of Params":"25M","Top 1 Accuracy":"65.9%"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-1","task":"Semi-Supervised Image Classification","dataset":"ImageNet - 1% labeled data","model":"PCL (ResNet-50)","rank_in_archive_order":55,"of":65,"metrics":{"Top 5 Accuracy":"75.6%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.04966","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.04966"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salesforce/MoPro","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salesforce/PCL","reach":null}],"summary":{"ran_violates":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"ebb82e479d372d1d","entry":"calculate_ap","repo":"salesforce/PCL","repo_kind":"official","path":"eval_svm_voc.py","file_url":"https://github.com/salesforce/PCL/blob/HEAD/eval_svm_voc.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebb82e479d372d1d"}},{"code_sha256_prefix":"5779ac02cf555fe2","entry":"get_precision_recall","repo":"salesforce/PCL","repo_kind":"official","path":"eval_svm_voc.py","file_url":"https://github.com/salesforce/PCL/blob/HEAD/eval_svm_voc.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5779ac02cf555fe2"}},{"code_sha256_prefix":"9b8289076669fe4f","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"9b8289076669fe4f"}},{"code_sha256_prefix":"2e8c79927398fd28","entry":"compute_features","repo":"salesforce/PCL","repo_kind":"official","path":"main_pcl.py","file_url":"https://github.com/salesforce/PCL/blob/HEAD/main_pcl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e8c79927398fd28"}},{"code_sha256_prefix":"e905656ff0c6e04a","entry":"validate","repo":"salesforce/PCL","repo_kind":"official","path":"eval_cls_imagenet.py","file_url":"https://github.com/salesforce/PCL/blob/HEAD/eval_cls_imagenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e905656ff0c6e04a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}