{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-llms-implicitly-exhibit-user","title":"A Study of Implicit Ranking Unfairness in Large Language Models","arxiv_id":"2311.07054","date":"2023-11-13","proceeding":null,"authors":["Chen Xu","Wenjie Wang","Yuxin Li","Liang Pang","Jun Xu","Tat-Seng Chua"],"abstract":"Recently, Large Language Models (LLMs) have demonstrated a superior ability to serve as ranking models. However, concerns have arisen as LLMs will exhibit discriminatory ranking behaviors based on users' sensitive attributes (\\eg gender). Worse still, in this paper, we identify a subtler form of discrimination in LLMs, termed \\textit{implicit ranking unfairness}, where LLMs exhibit discriminatory ranking patterns based solely on non-sensitive user profiles, such as user names. Such implicit unfairness is more widespread but less noticeable, threatening the ethical foundation. To comprehensively explore such unfairness, our analysis will focus on three research aspects: (1) We propose an evaluation method to investigate the severity of implicit ranking unfairness. (2) We uncover the reasons for causing such unfairness. (3) To mitigate such unfairness effectively, we utilize a pair-wise regression method to conduct fair-aware data augmentation for LLM fine-tuning. The experiment demonstrates that our method outperforms existing approaches in ranking fairness, achieving this with only a small reduction in accuracy. Lastly, we emphasize the need for the community to identify and mitigate the implicit unfairness, aiming to avert the potential deterioration in the reinforced human-LLMs ecosystem deterioration.","url_abs":"https://arxiv.org/abs/2311.07054v2","url_pdf":"https://arxiv.org/pdf/2311.07054v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-llms-implicitly-exhibit-user","repo_url":"https://github.com/xuchen0427/implicit_rank_unfairness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.07054","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.07054"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xuchen0427/implicit_rank_unfairness","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"0bbf001fcfa34ea0","entry":"PrepareDataForPairProbing","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"prepare_data.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/prepare_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0bbf001fcfa34ea0"}},{"code_sha256_prefix":"95fa5037e486fe3e","entry":"PrepareDataForPointProbing","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"prepare_data.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/prepare_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95fa5037e486fe3e"}},{"code_sha256_prefix":"029e43dc002f78a4","entry":"api_endpoint_from_url","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"API/openai.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/API/openai.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"029e43dc002f78a4"}},{"code_sha256_prefix":"a23e38819756489b","entry":"load_pair_model","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"rank_model.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/rank_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a23e38819756489b"}},{"code_sha256_prefix":"61e49605fd4ef1cc","entry":"load_point_model","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"rank_model.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/rank_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61e49605fd4ef1cc"}},{"code_sha256_prefix":"b0d8c94254fea6bb","entry":"read_embs_from_file","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"prepare_data.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/prepare_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0d8c94254fea6bb"}},{"code_sha256_prefix":"31b0d67913003d72","entry":"generate_prompt","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"train_llama.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/train_llama.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31b0d67913003d72"}},{"code_sha256_prefix":"766df5c74af871f7","entry":"num_tokens_consumed_from_request","repo":"xuchen0427/implicit_rank_unfairness","repo_kind":"official","path":"API/openai.py","file_url":"https://github.com/xuchen0427/implicit_rank_unfairness/blob/HEAD/API/openai.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"766df5c74af871f7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}