{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-empirical-study-on-learning-fairness","title":"An Empirical Study on Learning Fairness Metrics for COMPAS Data with Human Supervision","arxiv_id":"1910.10255","date":"2019-10-22","proceeding":null,"authors":["Hanchen Wang","Nina Grgic-Hlaca","Preethi Lahoti","Krishna P. Gummadi","Adrian Weller"],"abstract":"The notion of individual fairness requires that similar people receive similar treatment. However, this is hard to achieve in practice since it is difficult to specify the appropriate similarity metric. In this work, we attempt to learn such similarity metric from human annotated data. We gather a new dataset of human judgments on a criminal recidivism prediction (COMPAS) task. By assuming the human supervision obeys the principle of individual fairness, we leverage prior work on metric learning, evaluate the performance of several metric learning methods on our dataset, and show that the learned metrics outperform the Euclidean and Precision metric under various criteria. We do not provide a way to directly learn a similarity metric satisfying the individual fairness, but to provide an empirical study on how to derive the similarity metric from human supervisors, then future work can use this as a tool to understand human supervision.","url_abs":"https://arxiv.org/abs/1910.10255v2","url_pdf":"https://arxiv.org/pdf/1910.10255v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-empirical-study-on-learning-fairness","repo_url":"https://github.com/hansen7/LearnFairMetric_Empirical","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"metric-learning","task_name":"Metric Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.10255","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.10255"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hansen7/LearnFairMetric_Empirical","reach":null}],"summary":{"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"70c03d4b573cbdd0","entry":"load_dataset_hcon","repo":"hansen7/LearnFairMetric_Empirical","repo_kind":"official","path":"code/my_LSML_.py","file_url":"https://github.com/hansen7/LearnFairMetric_Empirical/blob/HEAD/code/my_LSML_.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"70c03d4b573cbdd0"}},{"code_sha256_prefix":"f6d3e5b75f3b7538","entry":"one_hot_vec","repo":"hansen7/LearnFairMetric_Empirical","repo_kind":"official","path":"code/my_LSML_.py","file_url":"https://github.com/hansen7/LearnFairMetric_Empirical/blob/HEAD/code/my_LSML_.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f6d3e5b75f3b7538"}},{"code_sha256_prefix":"b1a1e755e629ee5d","entry":"quadruplets_gen","repo":"hansen7/LearnFairMetric_Empirical","repo_kind":"official","path":"code/my_LSML_.py","file_url":"https://github.com/hansen7/LearnFairMetric_Empirical/blob/HEAD/code/my_LSML_.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1a1e755e629ee5d"}},{"code_sha256_prefix":"9ab3735b708c7ca6","entry":"similarity_edges_from_metric_by_knn_hcon","repo":"hansen7/LearnFairMetric_Empirical","repo_kind":"official","path":"code/Similarity_Edges.py","file_url":"https://github.com/hansen7/LearnFairMetric_Empirical/blob/HEAD/code/Similarity_Edges.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ab3735b708c7ca6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}