{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-11974","title":"Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization","arxiv_id":"2605.11974","date":"2026-05-12","proceeding":null,"authors":["Xu Chu","Guanyu Wang","Zhijie Tan","Xinrong Chen","Ziyu Li","Tong Mo","Weiping Li"],"abstract":"Large Language Models (LLMs) suffer from order bias, where their performance is affected by the arrangement order of input elements. This unfairness limits the model's applications in scenarios such as in-context learning and Retrieval-Augmented Generation (RAG). Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model's inherent order bias unresolved. Other studies mitigate order sensitivity through supervised fine-tuning using augmented training sets with multiple order variants, but often at the cost of accuracy, trapping the model in consistent yet incorrect hallucinations. In this paper, we propose \\textbf{D}ual \\textbf{G}roup \\textbf{A}dvantage \\textbf{O}ptimization (\\textbf{DGAO}), which aims to improve model accuracy and order stability simultaneously. DGAO calculates and balances intra-group relative accuracy advantage and inter-group relative stability advantage, rewarding the policy model for generating order-stable and correct outputs while penalizing order-sensitive or incorrect responses. This marks the first time reinforcement learning has been used to mitigate LLMs' order sensitivity. We also propose two new metrics, Consistency Rate and Overconfidence Rate, to reveal the pseudo-stability of previous methods and guide more comprehensive evaluation. Extensive experiments demonstrate that DGAO achieves superior order fairness while improving performance on RAG, mathematical reasoning, and classification tasks. Our code is available at: https://github.com/Hyalinesky/DGAO.","url_abs":"https://arxiv.org/abs/2605.11974","url_pdf":"https://arxiv.org/pdf/2605.11974","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.11974","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.11974"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Hyalinesky/DGAO","reach":null}],"summary":{"ran":9,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":15,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c70072e46530e6e1","entry":"apply_chat_template","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/data_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c70072e46530e6e1"}},{"code_sha256_prefix":"8f1ce89b816a0734","entry":"convert_state_dict","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/models/sd_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/models/sd_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f1ce89b816a0734"}},{"code_sha256_prefix":"dfd0ecd9cb25af69","entry":"flatten_dict","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/core.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfd0ecd9cb25af69"}},{"code_sha256_prefix":"5439c57a72337fa8","entry":"is_conversational","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/data_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5439c57a72337fa8"}},{"code_sha256_prefix":"4f7075aac2163cbc","entry":"masked_mean","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/core.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f7075aac2163cbc"}},{"code_sha256_prefix":"67607956afbb2a83","entry":"masked_var","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/core.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"67607956afbb2a83"}},{"code_sha256_prefix":"2d343f4f4297efa7","entry":"maybe_apply_chat_template","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/data_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d343f4f4297efa7"}},{"code_sha256_prefix":"a6cece2ffebe0f9c","entry":"nanstd","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/trainer/dgao_trainer.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/trainer/dgao_trainer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a6cece2ffebe0f9c"}},{"code_sha256_prefix":"4267969b53182b8f","entry":"shuffle_tensor_dict","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/trainer/dgao_trainer.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/trainer/dgao_trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4267969b53182b8f"}},{"code_sha256_prefix":"ba58cd6dfd66f89c","entry":"split_tensor_dict","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/trainer/dgao_trainer.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/trainer/dgao_trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba58cd6dfd66f89c"}},{"code_sha256_prefix":"7bdaf561b7b05114","entry":"aesthetic_scorer","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/models/auxiliary_modules.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/models/auxiliary_modules.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7bdaf561b7b05114"}},{"code_sha256_prefix":"fa59c920caf016e6","entry":"convert_state_dict_to_diffusers","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/models/sd_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/models/sd_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa59c920caf016e6"}},{"code_sha256_prefix":"f230a287da3ef006","entry":"get_act_offloading_ctx_manager","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/models/activation_offloading.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/models/activation_offloading.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f230a287da3ef006"}},{"code_sha256_prefix":"48cdac62e24819f1","entry":"is_liger_kernel_available","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/import_utils.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/import_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48cdac62e24819f1"}},{"code_sha256_prefix":"2ad63a9a0bae9859","entry":"scheduler_step","repo":"Hyalinesky/DGAO","repo_kind":"found_in_text","path":"trl/models/modeling_sd_base.py","file_url":"https://github.com/Hyalinesky/DGAO/blob/HEAD/trl/models/modeling_sd_base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ad63a9a0bae9859"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":2,"rejected_by_independent_check":0,"refused_by_a_rule":2,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}