{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/human-guided-fair-classification-for-natural","title":"Human-Guided Fair Classification for Natural Language Processing","arxiv_id":"2212.10154","date":"2022-12-20","proceeding":null,"authors":["Florian E. Dorner","Momchil Peychev","Nikola Konstantinov","Naman Goel","Elliott Ash","Martin Vechev"],"abstract":"Text classifiers have promising applications in high-stake tasks such as resume screening and content moderation. These classifiers must be fair and avoid discriminatory decisions by being invariant to perturbations of sensitive attributes such as gender or ethnicity. However, there is a gap between human intuition about these perturbations and the formal similarity specifications capturing them. While existing research has started to address this gap, current methods are based on hardcoded word replacements, resulting in specifications with limited expressivity or ones that fail to fully align with human intuition (e.g., in cases of asymmetric counterfactuals). This work proposes novel methods for bridging this gap by discovering expressive and intuitive individual fairness specifications. We show how to leverage unsupervised style transfer and GPT-3's zero-shot capabilities to automatically generate expressive candidate pairs of semantically similar sentences that differ along sensitive attributes. We then validate the generated pairs via an extensive crowdsourcing study, which confirms that a lot of these pairs align with human intuition about fairness in the context of toxicity classification. Finally, we show how limited amounts of human feedback can be leveraged to learn a similarity specification that can be used to train downstream fairness-aware models.","url_abs":"https://arxiv.org/abs/2212.10154v2","url_pdf":"https://arxiv.org/pdf/2212.10154v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"human-guided-fair-classification-for-natural","repo_url":"https://github.com/eth-sri/fairness-feedback-nlp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"CC0-1.0"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"style-transfer","task_name":"Style Transfer"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.10154","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.10154"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/eth-sri/fairness-feedback-nlp","reach":{"status":"ok","spdx":"CC0-1.0"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ea99d6212f1a6d27","entry":"get_agnostic_few_shot_prompt","repo":"eth-sri/fairness-feedback-nlp","repo_kind":"official","path":"Code/Transfer/Datasets/Attribute_Transfer.py","file_url":"https://github.com/eth-sri/fairness-feedback-nlp/blob/HEAD/Code/Transfer/Datasets/Attribute_Transfer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea99d6212f1a6d27"}},{"code_sha256_prefix":"247cc3d718be4aef","entry":"get_basic_gpt_prompt","repo":"eth-sri/fairness-feedback-nlp","repo_kind":"official","path":"Code/Transfer/Datasets/Attribute_Transfer.py","file_url":"https://github.com/eth-sri/fairness-feedback-nlp/blob/HEAD/Code/Transfer/Datasets/Attribute_Transfer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"247cc3d718be4aef"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}