{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/regretful-decisions-under-label-noise","title":"Regretful Decisions under Label Noise","arxiv_id":"2504.09330","date":"2025-04-12","proceeding":null,"authors":["Sujay Nagaraj","Yang Liu","Flavio P. Calmon","Berk Ustun"],"abstract":"Machine learning models are routinely used to support decisions that affect individuals -- be it to screen a patient for a serious illness or to gauge their response to treatment. In these tasks, we are limited to learning models from datasets with noisy labels. In this paper, we study the instance-level impact of learning under label noise. We introduce a notion of regret for this regime which measures the number of unforeseen mistakes due to noisy labels. We show that standard approaches to learning under label noise can return models that perform well at a population level while subjecting individuals to a lottery of mistakes. We present a versatile approach to estimate the likelihood of mistakes at the individual level from a noisy dataset by training models over plausible realizations of datasets without label noise. This is supported by a comprehensive empirical study of label noise in clinical prediction tasks. Our results reveal how failure to anticipate mistakes can compromise model reliability and adoption, and demonstrate how we can address these challenges by anticipating and avoiding regretful decisions.","url_abs":"https://arxiv.org/abs/2504.09330v1","url_pdf":"https://arxiv.org/pdf/2504.09330v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"regretful-decisions-under-label-noise","repo_url":"https://github.com/sujaynagaraj/regretful_decisions","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.09330","atlas_url":"https://app.syntology.ai/?focus=2504.09330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.09330"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sujaynagaraj/regretful_decisions","reach":null}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"583844d3ecce86a9","entry":"abstain_order","repo":"sujaynagaraj/regretful_decisions","repo_kind":"official","path":"src/abstain.py","file_url":"https://github.com/sujaynagaraj/regretful_decisions/blob/HEAD/src/abstain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"583844d3ecce86a9"}},{"code_sha256_prefix":"486bb551ac48075c","entry":"get_uncertainty","repo":"sujaynagaraj/regretful_decisions","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/sujaynagaraj/regretful_decisions/blob/HEAD/src/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"486bb551ac48075c"}},{"code_sha256_prefix":"5b12f2f9fc0e18b3","entry":"load_abstain","repo":"sujaynagaraj/regretful_decisions","repo_kind":"official","path":"src/abstain.py","file_url":"https://github.com/sujaynagaraj/regretful_decisions/blob/HEAD/src/abstain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b12f2f9fc0e18b3"}},{"code_sha256_prefix":"28879ae49feb5341","entry":"run_procedure_abstain","repo":"sujaynagaraj/regretful_decisions","repo_kind":"official","path":"src/abstain.py","file_url":"https://github.com/sujaynagaraj/regretful_decisions/blob/HEAD/src/abstain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"28879ae49feb5341"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}