{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/probabilistic-contrastive-learning-for-long","title":"Probabilistic Contrastive Learning for Long-Tailed Visual Recognition","arxiv_id":"2403.06726","date":"2024-03-11","proceeding":null,"authors":["Chaoqun Du","Yulin Wang","Shiji Song","Gao Huang"],"abstract":"Long-tailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Such imbalance issue considerably impairs the performance of standard supervised learning algorithms, which are mainly designed for balanced training sets. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. However, the performance of supervised contrastive learning is plagued by an inherent challenge: it necessitates sufficiently large batches of training data to construct contrastive pairs that cover all categories, yet this requirement is difficult to meet in the context of class-imbalanced data. To overcome this obstacle, we propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space, and samples contrastive pairs accordingly. In fact, estimating the distributions of all classes using features in a small batch, particularly for imbalanced data, is not feasible. Our key idea is to introduce a reasonable and simple assumption that the normalized features in contrastive learning follow a mixture of von Mises-Fisher (vMF) distributions on unit space, which brings two-fold benefits. First, the distribution parameters can be estimated using only the first sample moment, which can be efficiently computed in an online manner across different batches. Second, based on the estimated distribution, the vMF distribution allows us to sample an infinite number of contrastive pairs and derive a closed form of the expected contrastive loss for efficient optimization. Our code is available at https://github.com/LeapLabTHU/ProCo.","url_abs":"https://arxiv.org/abs/2403.06726v2","url_pdf":"https://arxiv.org/pdf/2403.06726v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"probabilistic-contrastive-learning-for-long","repo_url":"https://github.com/leaplabthu/proco","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"long-tail-learning","task_name":"Long-tail Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/long-tail-learning-on-imagenet-lt","task":"Long-tail Learning","dataset":"ImageNet-LT","model":"ProCo (ResNet50)","rank_in_archive_order":14,"of":69,"metrics":{"Top-1 Accuracy":"60.2"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-imagenet-lt","task":"Long-tail Learning","dataset":"ImageNet-LT","model":"ProCo (ResNeXt50)","rank_in_archive_order":24,"of":69,"metrics":{"Top-1 Accuracy":"58.0"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-inaturalist-2018","task":"Long-tail Learning","dataset":"iNaturalist 2018","model":"ProCo (ResNet50)","rank_in_archive_order":10,"of":43,"metrics":{"Top-1 Accuracy":"75.8%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.06726","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.06726"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leaplabthu/proco","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c19231378e41dc5a","entry":"bool_flag","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/utils.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c19231378e41dc5a"}},{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/models/resnext.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/models/resnext.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/models/resnext.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/models/resnext.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"9bb315ea09b14ff5","entry":"miller_recurrence","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/loss/proco.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/loss/proco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9bb315ea09b14ff5"}},{"code_sha256_prefix":"3612da0362c505e5","entry":"shot_acc","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/utils.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3612da0362c505e5"}},{"code_sha256_prefix":"a4b471197204b414","entry":"translate_x_rel","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/randaugment.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a4b471197204b414"}},{"code_sha256_prefix":"7be5050a9402a4c8","entry":"resnet50","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/models/resnext.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/models/resnext.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7be5050a9402a4c8"}},{"code_sha256_prefix":"7b4e3d0bd192b677","entry":"shear_x","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/randaugment.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/randaugment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b4e3d0bd192b677"}},{"code_sha256_prefix":"e31c4b500a240e69","entry":"shear_y","repo":"leaplabthu/proco","repo_kind":"official","path":"ProCo/randaugment.py","file_url":"https://github.com/leaplabthu/proco/blob/HEAD/ProCo/randaugment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e31c4b500a240e69"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}