{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-empirical-study-of-rich-subgroup-fairness","title":"An Empirical Study of Rich Subgroup Fairness for Machine Learning","arxiv_id":"1808.08166","date":"2018-08-24","proceeding":null,"authors":["Michael Kearns","Seth Neel","Aaron Roth","Zhiwei Steven Wu"],"abstract":"Kearns et al. [2018] recently proposed a notion of rich subgroup fairness\nintended to bridge the gap between statistical and individual notions of\nfairness. Rich subgroup fairness picks a statistical fairness constraint (say,\nequalizing false positive rates across protected groups), but then asks that\nthis constraint hold over an exponentially or infinitely large collection of\nsubgroups defined by a class of functions with bounded VC dimension. They give\nan algorithm guaranteed to learn subject to this constraint, under the\ncondition that it has access to oracles for perfectly learning absent a\nfairness constraint. In this paper, we undertake an extensive empirical\nevaluation of the algorithm of Kearns et al. On four real datasets for which\nfairness is a concern, we investigate the basic convergence of the algorithm\nwhen instantiated with fast heuristics in place of learning oracles, measure\nthe tradeoffs between fairness and accuracy, and compare this approach with the\nrecent algorithm of Agarwal et al. [2018], which implements weaker and more\ntraditional marginal fairness constraints defined by individual protected\nattributes. We find that in general, the Kearns et al. algorithm converges\nquickly, large gains in fairness can be obtained with mild costs to accuracy,\nand that optimizing accuracy subject only to marginal fairness leads to\nclassifiers with substantial subgroup unfairness. We also provide a number of\nanalyses and visualizations of the dynamics and behavior of the Kearns et al.\nalgorithm. Overall we find this algorithm to be effective on real data, and\nrich subgroup fairness to be a viable notion in practice.","url_abs":"http://arxiv.org/abs/1808.08166v1","url_pdf":"http://arxiv.org/pdf/1808.08166v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-empirical-study-of-rich-subgroup-fairness","repo_url":"https://github.com/SaeedSharifiMa/FairDP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"an-empirical-study-of-rich-subgroup-fairness","repo_url":"https://github.com/algowatchpenn/GerryFair","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"an-empirical-study-of-rich-subgroup-fairness","repo_url":"https://github.com/lacava/fair_gp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"an-empirical-study-of-rich-subgroup-fairness","repo_url":"https://github.com/sethneel/GerryFair","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"an-empirical-study-of-rich-subgroup-fairness","repo_url":"https://github.com/wibrown/GerryFair","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1808.08166","atlas_url":"https://app.syntology.ai/?focus=1808.08166","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1808.08166"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lacava/fair_gp","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/algowatchpenn/GerryFair","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wibrown/GerryFair","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SaeedSharifiMa/FairDP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sethneel/GerryFair","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"unverified":9},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"b2afbc4ac64576bb","entry":"gen_a","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b2afbc4ac64576bb"}},{"code_sha256_prefix":"791852801122e9e5","entry":"learner_costs","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"791852801122e9e5"}},{"code_sha256_prefix":"d475d39df96fac45","entry":"calc_disp","repo":"algowatchpenn/GerryFair","repo_kind":"listed","path":"gerryfair/heatmap.py","file_url":"https://github.com/algowatchpenn/GerryFair/blob/HEAD/gerryfair/heatmap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d475d39df96fac45"}},{"code_sha256_prefix":"bf099d646e73cf6e","entry":"center","repo":"algowatchpenn/GerryFair","repo_kind":"listed","path":"gerryfair/clean.py","file_url":"https://github.com/algowatchpenn/GerryFair/blob/HEAD/gerryfair/clean.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf099d646e73cf6e"}},{"code_sha256_prefix":"cd537de833dfde8f","entry":"clean_dataset","repo":"algowatchpenn/GerryFair","repo_kind":"listed","path":"gerryfair/clean.py","file_url":"https://github.com/algowatchpenn/GerryFair/blob/HEAD/gerryfair/clean.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd537de833dfde8f"}},{"code_sha256_prefix":"8adf5d639ebb5df1","entry":"clean_dataset","repo":"sethneel/GerryFair","repo_kind":"listed","path":"gerryfair/clean.py","file_url":"https://github.com/sethneel/GerryFair/blob/HEAD/gerryfair/clean.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8adf5d639ebb5df1"}},{"code_sha256_prefix":"e1efab7d887f28a5","entry":"get_fp","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e1efab7d887f28a5"}},{"code_sha256_prefix":"bedf19c17f580dd6","entry":"get_group","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"bedf19c17f580dd6"}},{"code_sha256_prefix":"f9a1bd2d964a57e2","entry":"heat_map","repo":"algowatchpenn/GerryFair","repo_kind":"listed","path":"gerryfair/heatmap.py","file_url":"https://github.com/algowatchpenn/GerryFair/blob/HEAD/gerryfair/heatmap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9a1bd2d964a57e2"}},{"code_sha256_prefix":"97207ff93fd80888","entry":"one_hot_code","repo":"algowatchpenn/GerryFair","repo_kind":"listed","path":"gerryfair/clean.py","file_url":"https://github.com/algowatchpenn/GerryFair/blob/HEAD/gerryfair/clean.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97207ff93fd80888"}},{"code_sha256_prefix":"d17910bf14c509ca","entry":"plot_pareto","repo":"sethneel/GerryFair","repo_kind":"listed","path":"gerryfair/fairness_plots.py","file_url":"https://github.com/sethneel/GerryFair/blob/HEAD/gerryfair/fairness_plots.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d17910bf14c509ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}