{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-agnostic-characterization-of-fairness","title":"FACT: A Diagnostic for Group Fairness Trade-offs","arxiv_id":"2004.03424","date":"2020-04-07","proceeding":null,"authors":["Joon Sik Kim","Jiahao Chen","Ameet Talwalkar"],"abstract":"Group fairness, a class of fairness notions that measure how different groups of individuals are treated differently according to their protected attributes, has been shown to conflict with one another, often with a necessary cost in loss of model's predictive performance. We propose a general diagnostic that enables systematic characterization of these trade-offs in group fairness. We observe that the majority of group fairness notions can be expressed via the fairness-confusion tensor, which is the confusion matrix split according to the protected attribute values. We frame several optimization problems that directly optimize both accuracy and fairness objectives over the elements of this tensor, which yield a general perspective for understanding multiple trade-offs including group fairness incompatibilities. It also suggests an alternate post-processing method for designing fair classifiers. On synthetic and real datasets, we demonstrate the use cases of our diagnostic, particularly on understanding the trade-off landscape between accuracy and fairness.","url_abs":"https://arxiv.org/abs/2004.03424v3","url_pdf":"https://arxiv.org/pdf/2004.03424v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-agnostic-characterization-of-fairness","repo_url":"https://github.com/wnstlr/FACT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2004.03424","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.03424"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wnstlr/FACT","reach":null}],"summary":{"ran_draft_wrong":4,"ran_fixture":2,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"86a3413b79c196ba","entry":"compute_stats","repo":"wnstlr/fact","repo_kind":"official","path":"FACT/fairness.py","file_url":"https://github.com/wnstlr/fact/blob/HEAD/FACT/fairness.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86a3413b79c196ba"}},{"code_sha256_prefix":"088e34561105e9a3","entry":"eps_opt","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"088e34561105e9a3"}},{"code_sha256_prefix":"df76b86dee554f60","entry":"get_fairness_mats","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df76b86dee554f60"}},{"code_sha256_prefix":"80b808c8eacf1735","entry":"opt_fairness_const_perf","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80b808c8eacf1735"}},{"code_sha256_prefix":"3fc04c307fe98d96","entry":"solve_LAFOP","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fc04c307fe98d96"}},{"code_sha256_prefix":"b074bde51cf49aca","entry":"solve_PFOP","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b074bde51cf49aca"}},{"code_sha256_prefix":"75c17422b5e1aea4","entry":"compute_base_rates","repo":"wnstlr/fact","repo_kind":"official","path":"FACT/fairness.py","file_url":"https://github.com/wnstlr/fact/blob/HEAD/FACT/fairness.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75c17422b5e1aea4"}},{"code_sha256_prefix":"25accd8e7f50eaac","entry":"test_fair_instance","repo":"wnstlr/FACT","repo_kind":"official","path":"FACT/lin_opt.py","file_url":"https://github.com/wnstlr/FACT/blob/HEAD/FACT/lin_opt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25accd8e7f50eaac"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}