{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/certifying-and-removing-disparate-impact","title":"Certifying and removing disparate impact","arxiv_id":"1412.3756","date":"2014-12-11","proceeding":null,"authors":["Michael Feldman","Sorelle Friedler","John Moeller","Carlos Scheidegger","Suresh Venkatasubramanian"],"abstract":"What does it mean for an algorithm to be biased? In U.S. law, unintentional\nbias is encoded via disparate impact, which occurs when a selection process has\nwidely different outcomes for different groups, even as it appears to be\nneutral. This legal determination hinges on a definition of a protected class\n(ethnicity, gender, religious practice) and an explicit description of the\nprocess.\n  When the process is implemented using computers, determining disparate impact\n(and hence bias) is harder. It might not be possible to disclose the process.\nIn addition, even if the process is open, it might be hard to elucidate in a\nlegal setting how the algorithm makes its decisions. Instead of requiring\naccess to the algorithm, we propose making inferences based on the data the\nalgorithm uses.\n  We make four contributions to this problem. First, we link the legal notion\nof disparate impact to a measure of classification accuracy that while known,\nhas received relatively little attention. Second, we propose a test for\ndisparate impact based on analyzing the information leakage of the protected\nclass from the other data attributes. Third, we describe methods by which data\nmight be made unbiased. Finally, we present empirical evidence supporting the\neffectiveness of our test for disparate impact and our approach for both\nmasking bias and preserving relevant information in the data. Interestingly,\nour approach resembles some actual selection practices that have recently\nreceived legal scrutiny.","url_abs":"http://arxiv.org/abs/1412.3756v3","url_pdf":"http://arxiv.org/pdf/1412.3756v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"certifying-and-removing-disparate-impact","repo_url":"https://github.com/algofairness/BlackBoxAuditing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"certifying-and-removing-disparate-impact","repo_url":"https://github.com/cfalk/BlackBoxAuditing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1412.3756","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}