{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-fairness-and-calibration","title":"On Fairness and Calibration","arxiv_id":"1709.02012","date":"2017-09-06","proceeding":"NeurIPS 2017 12","authors":["Geoff Pleiss","Manish Raghavan","Felix Wu","Jon Kleinberg","Kilian Q. Weinberger"],"abstract":"The machine learning community has become increasingly concerned with the\npotential for bias and discrimination in predictive models. This has motivated\na growing line of work on what it means for a classification procedure to be\n\"fair.\" In this paper, we investigate the tension between minimizing error\ndisparity across different population groups while maintaining calibrated\nprobability estimates. We show that calibration is compatible only with a\nsingle error constraint (i.e. equal false-negatives rates across groups), and\nshow that any algorithm that satisfies this relaxation is no better than\nrandomizing a percentage of predictions for an existing classifier. These\nunsettling findings, which extend and generalize existing results, are\nempirically confirmed on several datasets.","url_abs":"http://arxiv.org/abs/1709.02012v2","url_pdf":"http://arxiv.org/pdf/1709.02012v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-fairness-and-calibration","repo_url":"https://github.com/gpleiss/equalized_odds_and_calibration","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1709.02012","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}