{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/social-bias-meets-data-bias-the-impacts-of","title":"Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria","arxiv_id":"2206.00137","date":"2022-05-31","proceeding":null,"authors":["Yiqiao Liao","Parinaz Naghizadeh"],"abstract":"Although many fairness criteria have been proposed to ensure that machine learning algorithms do not exhibit or amplify our existing social biases, these algorithms are trained on datasets that can themselves be statistically biased. In this paper, we investigate the robustness of a number of existing (demographic) fairness criteria when the algorithm is trained on biased data. We consider two forms of dataset bias: errors by prior decision makers in the labeling process, and errors in measurement of the features of disadvantaged individuals. We analytically show that some constraints (such as Demographic Parity) can remain robust when facing certain statistical biases, while others (such as Equalized Odds) are significantly violated if trained on biased data. We also analyze the sensitivity of these criteria and the decision maker's utility to biases. We provide numerical experiments based on three real-world datasets (the FICO, Adult, and German credit score datasets) supporting our analytical findings. Our findings present an additional guideline for choosing among existing fairness criteria, or for proposing new criteria, when available datasets may be biased.","url_abs":"https://arxiv.org/abs/2206.00137v4","url_pdf":"https://arxiv.org/pdf/2206.00137v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"social-bias-meets-data-bias-the-impacts-of","repo_url":"https://github.com/yl489/social-bias-meets-data-bias","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"credit-score","task_name":"Credit score"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.00137","atlas_url":"https://app.syntology.ai/?focus=2206.00137","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.00137"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yl489/social-bias-meets-data-bias","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6350a088a64f5069","entry":"cleanup_frame","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_util.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6350a088a64f5069"}},{"code_sha256_prefix":"0818fbb6e5a2b7dd","entry":"compute_rate_lists","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0818fbb6e5a2b7dd"}},{"code_sha256_prefix":"aedcb3f910b45e3d","entry":"compute_rates_lists","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"synthetic_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/synthetic_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aedcb3f910b45e3d"}},{"code_sha256_prefix":"474c76619d591387","entry":"flip_labels","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"adult_german_util.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/adult_german_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"474c76619d591387"}},{"code_sha256_prefix":"601a98f39e06ebd3","entry":"get_pdf_from_cdf","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"601a98f39e06ebd3"}},{"code_sha256_prefix":"38c5dcc898d3bb0d","entry":"get_util","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"synthetic_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/synthetic_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38c5dcc898d3bb0d"}},{"code_sha256_prefix":"d75ebdb3773b08a9","entry":"get_util_by_thresh_ind","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d75ebdb3773b08a9"}},{"code_sha256_prefix":"65c9ec14c8dc5cfb","entry":"read_totals","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_util.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65c9ec14c8dc5cfb"}},{"code_sha256_prefix":"835426060077f516","entry":"theta_MU","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"synthetic_solver.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/synthetic_solver.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"835426060077f516"}},{"code_sha256_prefix":"c2824956a3f0b6ed","entry":"convert_percentiles","repo":"yl489/social-bias-meets-data-bias","repo_kind":"official","path":"fico_util.py","file_url":"https://github.com/yl489/social-bias-meets-data-bias/blob/HEAD/fico_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2824956a3f0b6ed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}