{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-soft-classifiers-to-hard-decisions-how","title":"From Soft Classifiers to Hard Decisions: How fair can we be?","arxiv_id":"1810.02003","date":"2018-10-03","proceeding":null,"authors":["Ran Canetti","Aloni Cohen","Nishanth Dikkala","Govind Ramnarayan","Sarah Scheffler","Adam Smith"],"abstract":"A popular methodology for building binary decision-making classifiers in the\npresence of imperfect information is to first construct a non-binary \"scoring\"\nclassifier that is calibrated over all protected groups, and then to\npost-process this score to obtain a binary decision. We study the feasibility\nof achieving various fairness properties by post-processing calibrated scores,\nand then show that deferring post-processors allow for more fairness conditions\nto hold on the final decision. Specifically, we show:\n  1. There does not exist a general way to post-process a calibrated classifier\nto equalize protected groups' positive or negative predictive value (PPV or\nNPV). For certain \"nice\" calibrated classifiers, either PPV or NPV can be\nequalized when the post-processor uses different thresholds across protected\ngroups, though there exist distributions of calibrated scores for which the two\nmeasures cannot be both equalized. When the post-processing consists of a\nsingle global threshold across all groups, natural fairness properties, such as\nequalizing PPV in a nontrivial way, do not hold even for \"nice\" classifiers.\n  2. When the post-processing is allowed to `defer' on some decisions (that is,\nto avoid making a decision by handing off some examples to a separate process),\nthen for the non-deferred decisions, the resulting classifier can be made to\nequalize PPV, NPV, false positive rate (FPR) and false negative rate (FNR)\nacross the protected groups. This suggests a way to partially evade the\nimpossibility results of Chouldechova and Kleinberg et al., which preclude\nequalizing all of these measures simultaneously. We also present different\ndeferring strategies and show how they affect the fairness properties of the\noverall system.\n  We evaluate our post-processing techniques using the COMPAS data set from\n2016.","url_abs":"http://arxiv.org/abs/1810.02003v2","url_pdf":"http://arxiv.org/pdf/1810.02003v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-soft-classifiers-to-hard-decisions-how","repo_url":"https://github.com/nishanthdikkala/postprocessing-deferrals","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1810.02003","atlas_url":"https://app.syntology.ai/?focus=1810.02003","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}