{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automated-directed-fairness-testing","title":"Automated Directed Fairness Testing","arxiv_id":"1807.00468","date":"2018-07-02","proceeding":null,"authors":["Sakshi Udeshi","Pryanshu Arora","Sudipta Chattopadhyay"],"abstract":"Fairness is a critical trait in decision making. As machine-learning models\nare increasingly being used in sensitive application domains (e.g. education\nand employment) for decision making, it is crucial that the decisions computed\nby such models are free of unintended bias. But how can we automatically\nvalidate the fairness of arbitrary machine-learning models? For a given\nmachine-learning model and a set of sensitive input parameters, our AEQUITAS\napproach automatically discovers discriminatory inputs that highlight fairness\nviolation. At the core of AEQUITAS are three novel strategies to employ\nprobabilistic search over the input space with the objective of uncovering\nfairness violation. Our AEQUITAS approach leverages inherent robustness\nproperty in common machine-learning models to design and implement scalable\ntest generation methodologies. An appealing feature of our generated test\ninputs is that they can be systematically added to the training set of the\nunderlying model and improve its fairness. To this end, we design a fully\nautomated module that guarantees to improve the fairness of the underlying\nmodel.\n  We implemented AEQUITAS and we have evaluated it on six state-of-the-art\nclassifiers, including a classifier that was designed with fairness\nconstraints. We show that AEQUITAS effectively generates inputs to uncover\nfairness violation in all the subject classifiers and systematically improves\nthe fairness of the respective models using the generated test inputs. In our\nevaluation, AEQUITAS generates up to 70% discriminatory inputs (w.r.t. the\ntotal number of inputs generated) and leverages these inputs to improve the\nfairness up to 94%.","url_abs":"http://arxiv.org/abs/1807.00468v2","url_pdf":"http://arxiv.org/pdf/1807.00468v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automated-directed-fairness-testing","repo_url":"https://github.com/sakshiudeshi/Aequitas","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}