{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adept-a-debiasing-prompt-framework","title":"ADEPT: A DEbiasing PrompT Framework","arxiv_id":"2211.05414","date":"2022-11-10","proceeding":null,"authors":["Ke Yang","Charles Yu","Yi Fung","Manling Li","Heng Ji"],"abstract":"Several works have proven that finetuning is an applicable approach for debiasing contextualized word embeddings. Similarly, discrete prompts with semantic meanings have shown to be effective in debiasing tasks. With unfixed mathematical representation at the token level, continuous prompts usually surpass discrete ones at providing a pre-trained language model (PLM) with additional task-specific information. Despite this, relatively few efforts have been made to debias PLMs by prompt tuning with continuous prompts compared to its discrete counterpart. Furthermore, for most debiasing methods that alter a PLM's original parameters, a major problem is the need to not only decrease the bias in the PLM but also to ensure that the PLM does not lose its representation ability. Finetuning methods typically have a hard time maintaining this balance, as they tend to violently remove meanings of attribute words. In this paper, we propose ADEPT, a method to debias PLMs using prompt tuning while maintaining the delicate balance between removing biases and ensuring representation ability. To achieve this, we propose a new training criterion inspired by manifold learning and equip it with an explicit debiasing term to optimize prompt tuning. In addition, we conduct several experiments with regard to the reliability, quality, and quantity of a previously proposed attribute training corpus in order to obtain a clearer prototype of a certain attribute, which indicates the attribute's position and relative distances to other words on the manifold. We evaluate ADEPT on several widely acknowledged debiasing benchmarks and downstream tasks, and find that it achieves competitive results while maintaining (and in some cases even improving) the PLM's representation ability. We further visualize words' correlation before and after debiasing a PLM, and give some possible explanations for the visible effects.","url_abs":"https://arxiv.org/abs/2211.05414v2","url_pdf":"https://arxiv.org/pdf/2211.05414v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adept-a-debiasing-prompt-framework","repo_url":"https://github.com/EmpathYang/ADEPT","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.05414","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.05414"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EmpathYang/ADEPT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran_fixture":1,"ran":4,"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"00b3090f89b698e1","entry":"build_relative_position","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/deberta.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/deberta.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00b3090f89b698e1"}},{"code_sha256_prefix":"ea37553c09194184","entry":"c2p_dynamic_expand","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/deberta.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/deberta.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea37553c09194184"}},{"code_sha256_prefix":"31501071e57d43c4","entry":"calculate_group_to_group_relative_distance_asymmetric","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"distance.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/distance.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31501071e57d43c4"}},{"code_sha256_prefix":"d6b5586516edd277","entry":"calculate_group_to_group_relative_distance_asymmetric_test","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"distance.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/distance.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6b5586516edd277"}},{"code_sha256_prefix":"d73ab432982f8191","entry":"calculate_group_to_one_relative_distance_asymmetric_test","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"distance.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/distance.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d73ab432982f8191"}},{"code_sha256_prefix":"5f516b6aaad9acb6","entry":"create_dataset","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"debias.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/debias.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f516b6aaad9acb6"}},{"code_sha256_prefix":"f46fa5e6d0948125","entry":"load_and_cache_examples","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"debias.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/debias.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f46fa5e6d0948125"}},{"code_sha256_prefix":"d0dd7e697cd0cfba","entry":"split_data","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"debias.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/debias.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0dd7e697cd0cfba"}},{"code_sha256_prefix":"0df39601dd0d5745","entry":"build_relative_position","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/debertaV2.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/debertaV2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0df39601dd0d5745"}},{"code_sha256_prefix":"05732f1a5126672d","entry":"get_mask","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/deberta.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/deberta.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05732f1a5126672d"}},{"code_sha256_prefix":"06439dc4fc5e69b0","entry":"get_model_deprecated","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/utils.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06439dc4fc5e69b0"}},{"code_sha256_prefix":"29371b9c270911e1","entry":"make_log_bucket_position","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"model/debertaV2.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/model/debertaV2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29371b9c270911e1"}},{"code_sha256_prefix":"8caa8979bca6e6cd","entry":"prepare_tokenizer","repo":"EmpathYang/ADEPT","repo_kind":"official","path":"collect_sentences.py","file_url":"https://github.com/EmpathYang/ADEPT/blob/HEAD/collect_sentences.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8caa8979bca6e6cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}