{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multiclass-learning-from-noisy-labels-for-non","title":"Multiclass Learning from Noisy Labels for Non-decomposable Performance Measures","arxiv_id":"2402.01055","date":"2024-02-01","proceeding":null,"authors":["Mingyuan Zhang","Shivani Agarwal"],"abstract":"There has been much interest in recent years in learning good classifiers from data with noisy labels. Most work on learning from noisy labels has focused on standard loss-based performance measures. However, many machine learning problems require using non-decomposable performance measures which cannot be expressed as the expectation or sum of a loss on individual examples; these include for example the H-mean, Q-mean and G-mean in class imbalance settings, and the Micro $F_1$ in information retrieval. In this paper, we design algorithms to learn from noisy labels for two broad classes of multiclass non-decomposable performance measures, namely, monotonic convex and ratio-of-linear, which encompass all the above examples. Our work builds on the Frank-Wolfe and Bisection based methods of Narasimhan et al. (2015). In both cases, we develop noise-corrected versions of the algorithms under the widely studied family of class-conditional noise models. We provide regret (excess risk) bounds for our algorithms, establishing that even though they are trained on noisy data, they are Bayes consistent in the sense that their performance converges to the optimal performance w.r.t. the clean (non-noisy) distribution. Our experiments demonstrate the effectiveness of our algorithms in handling label noise.","url_abs":"https://arxiv.org/abs/2402.01055v3","url_pdf":"https://arxiv.org/pdf/2402.01055v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multiclass-learning-from-noisy-labels-for-non","repo_url":"https://github.com/moshimowang/noisy-labels-non-decomposable","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.01055","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.01055"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/moshimowang/noisy-labels-non-decomposable","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"0818c3a3bff414e2","entry":"build_file_name","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"noise-corrected-expt.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/noise-corrected-expt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0818c3a3bff414e2"}},{"code_sha256_prefix":"fdacb17c6ef7e0f7","entry":"corrected_fmeasure","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"algorithm/bisection.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/algorithm/bisection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fdacb17c6ef7e0f7"}},{"code_sha256_prefix":"bf917c10d4b97fa9","entry":"frank_wolfe_post","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"solvers/fwpost.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/solvers/fwpost.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf917c10d4b97fa9"}},{"code_sha256_prefix":"ac3ccd075b49ded2","entry":"get_confusion_matrix","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"standard_funcs/confusion_matrix.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/standard_funcs/confusion_matrix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac3ccd075b49ded2"}},{"code_sha256_prefix":"73d39da0bab2bbed","entry":"get_confusion_matrix_from_loss_a","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"standard_funcs/confusion_matrix.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/standard_funcs/confusion_matrix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73d39da0bab2bbed"}},{"code_sha256_prefix":"8b9cb1a891766b19","entry":"get_confusion_matrix_threshold","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"standard_funcs/confusion_matrix.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/standard_funcs/confusion_matrix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b9cb1a891766b19"}},{"code_sha256_prefix":"b3d4f59c2390777a","entry":"robust","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"algorithm/models.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/algorithm/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3d4f59c2390777a"}},{"code_sha256_prefix":"1d783bd2d7c7f884","entry":"crossentropy","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"algorithm/models.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/algorithm/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1d783bd2d7c7f884"}},{"code_sha256_prefix":"12b0ba54e2dcb6e5","entry":"get_vec_eta","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"models/logistic_regression.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/models/logistic_regression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12b0ba54e2dcb6e5"}},{"code_sha256_prefix":"5e6af188c8b5f941","entry":"get_vec_eta_2","repo":"moshimowang/noisy-labels-non-decomposable","repo_kind":"official","path":"models/logistic_regression.py","file_url":"https://github.com/moshimowang/noisy-labels-non-decomposable/blob/HEAD/models/logistic_regression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e6af188c8b5f941"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}