{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrastive-learning-improves-model","title":"Contrastive Learning Improves Model Robustness Under Label Noise","arxiv_id":"2104.08984","date":"2021-04-19","proceeding":null,"authors":["Aritra Ghosh","Andrew Lan"],"abstract":"Deep neural network-based classifiers trained with the categorical cross-entropy (CCE) loss are sensitive to label noise in the training data. One common type of method that can mitigate the impact of label noise can be viewed as supervised robust methods; one can simply replace the CCE loss with a loss that is robust to label noise, or re-weight training samples and down-weight those with higher loss values. Recently, another type of method using semi-supervised learning (SSL) has been proposed, which augments these supervised robust methods to exploit (possibly) noisy samples more effectively. Although supervised robust methods perform well across different data types, they have been shown to be inferior to the SSL methods on image classification tasks under label noise. Therefore, it remains to be seen that whether these supervised robust methods can also perform well if they can utilize the unlabeled samples more effectively. In this paper, we show that by initializing supervised robust methods using representations learned through contrastive learning leads to significantly improved performance under label noise. Surprisingly, even the simplest method (training a classifier with the CCE loss) can outperform the state-of-the-art SSL method by more than 50\\% under high label noise when initialized with contrastive learning. Our implementation will be publicly available at {\\url{https://github.com/arghosh/noisy_label_pretrain}}.","url_abs":"https://arxiv.org/abs/2104.08984v1","url_pdf":"https://arxiv.org/pdf/2104.08984v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrastive-learning-improves-model","repo_url":"https://github.com/arghosh/noisy_label_pretrain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-clothing1m","task":"Image Classification","dataset":"Clothing1M","model":"MAE (SimCLR)","rank_in_archive_order":29,"of":51,"metrics":{"Accuracy":"73.36%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-clothing1m","task":"Image Classification","dataset":"Clothing1M","model":"Generalized CE (SimCLR)","rank_in_archive_order":30,"of":51,"metrics":{"Accuracy":"73.35%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-clothing1m","task":"Image Classification","dataset":"Clothing1M","model":"CCE (SimCLR)","rank_in_archive_order":32,"of":51,"metrics":{"Accuracy":"73.27%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.08984","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.08984"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arghosh/noisy_label_pretrain","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":12},"by_repo_kind":{"official":{"samples":13,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"11bc56dd2ecb7644","entry":"accuracy","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/models/losses.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11bc56dd2ecb7644"}},{"code_sha256_prefix":"0aeed6613e3f5540","entry":"asym_noise","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0aeed6613e3f5540"}},{"code_sha256_prefix":"961e370efdf679f8","entry":"data_loader","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"imagenet32.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/imagenet32.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"961e370efdf679f8"}},{"code_sha256_prefix":"3d5e11c2280cdb46","entry":"gather","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/models/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d5e11c2280cdb46"}},{"code_sha256_prefix":"e31c345c9535138f","entry":"get_config","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"scheduler.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/scheduler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e31c345c9535138f"}},{"code_sha256_prefix":"53f0e581017b7778","entry":"get_cpu","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"scheduler.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/scheduler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53f0e581017b7778"}},{"code_sha256_prefix":"9ef98e4eebd96977","entry":"get_memory","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"scheduler.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/scheduler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ef98e4eebd96977"}},{"code_sha256_prefix":"97860ae8a6484f60","entry":"load_train","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"imagenet32.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/imagenet32.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97860ae8a6484f60"}},{"code_sha256_prefix":"439f4fcb999995b5","entry":"mean_cumulative_gain","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/models/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"439f4fcb999995b5"}},{"code_sha256_prefix":"2d3453d13dc8c3f4","entry":"noise_matrix","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d3453d13dc8c3f4"}},{"code_sha256_prefix":"eab99ab911f1645a","entry":"pad","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"utils/datautils.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/utils/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eab99ab911f1645a"}},{"code_sha256_prefix":"ba6749b25e502f54","entry":"uniform_mix_C","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba6749b25e502f54"}},{"code_sha256_prefix":"4ebc0e09cb353eac","entry":"unpickle","repo":"arghosh/noisy_label_pretrain","repo_kind":"official","path":"imagenet32.py","file_url":"https://github.com/arghosh/noisy_label_pretrain/blob/HEAD/imagenet32.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ebc0e09cb353eac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}