{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/conformal-prediction-sets-can-cause-disparate","title":"Conformal Prediction Sets Can Cause Disparate Impact","arxiv_id":"2410.01888","date":"2024-10-02","proceeding":null,"authors":["Jesse C. Cresswell","Bhargava Kumar","Yi Sui","Mouloud Belbahri"],"abstract":"Although conformal prediction is a promising method for quantifying the uncertainty of machine learning models, the prediction sets it outputs are not inherently actionable. Many applications require a single output to act on, not several. To overcome this, prediction sets can be provided to a human who then makes an informed decision. In any such system it is crucial to ensure the fairness of outcomes across protected groups, and researchers have proposed that Equalized Coverage be used as the standard for fairness. By conducting experiments with human participants, we demonstrate that providing prediction sets can increase the unfairness of their decisions. Disquietingly, we find that providing sets that satisfy Equalized Coverage actually increases unfairness compared to marginal coverage. Instead of equalizing coverage, we propose to equalize set sizes across groups which empirically leads to more fair outcomes.","url_abs":"https://arxiv.org/abs/2410.01888v1","url_pdf":"https://arxiv.org/pdf/2410.01888v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"conformal-prediction-sets-can-cause-disparate","repo_url":"https://github.com/layer6ai-labs/conformal-prediction-fairness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conformal-prediction","task_name":"Conformal Prediction"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.01888","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01888"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/layer6ai-labs/conformal-prediction-fairness","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6731c12945128300","entry":"calibrate_avg_k","repo":"layer6ai-labs/conformal-prediction-fairness","repo_kind":"official","path":"average_k.py","file_url":"https://github.com/layer6ai-labs/conformal-prediction-fairness/blob/HEAD/average_k.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6731c12945128300"}},{"code_sha256_prefix":"3a66731c07d4d64b","entry":"get_average_k_sets","repo":"layer6ai-labs/conformal-prediction-fairness","repo_kind":"official","path":"average_k.py","file_url":"https://github.com/layer6ai-labs/conformal-prediction-fairness/blob/HEAD/average_k.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a66731c07d4d64b"}},{"code_sha256_prefix":"9ea2aadacd95e79c","entry":"get_config","repo":"layer6ai-labs/conformal-prediction-fairness","repo_kind":"official","path":"config.py","file_url":"https://github.com/layer6ai-labs/conformal-prediction-fairness/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ea2aadacd95e79c"}},{"code_sha256_prefix":"180793cf5c651f09","entry":"parse_config_arg","repo":"layer6ai-labs/conformal-prediction-fairness","repo_kind":"official","path":"config.py","file_url":"https://github.com/layer6ai-labs/conformal-prediction-fairness/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"180793cf5c651f09"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}