{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robustness-of-accuracy-metric-and-its","title":"Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels","arxiv_id":"2012.04193","date":"2020-12-08","proceeding":null,"authors":["Pengfei Chen","Junjie Ye","Guangyong Chen","Jingwei Zhao","Pheng-Ann Heng"],"abstract":"For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.","url_abs":"https://arxiv.org/abs/2012.04193v1","url_pdf":"https://arxiv.org/pdf/2012.04193v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robustness-of-accuracy-metric-and-its","repo_url":"https://github.com/chenpf1025/RobustnessAccuracy","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"learning-with-noisy-labels","task_name":"Learning with noisy labels"},{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"multi-class-classification","task_name":"Multi-class Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2012.04193","atlas_url":"https://app.syntology.ai/?focus=2012.04193","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}