{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reformatted-alignment","title":"Reformatted Alignment","arxiv_id":"2402.12219","date":"2024-02-19","proceeding":null,"authors":["Run-Ze Fan","Xuefeng Li","Haoyang Zou","Junlong Li","Shwai He","Ethan Chern","Jiewen Hu","PengFei Liu"],"abstract":"The quality of finetuning data is crucial for aligning large language models (LLMs) with human values. Current methods to improve data quality are either labor-intensive or prone to factual errors caused by LLM hallucinations. This paper explores elevating the quality of existing instruction data to better align with human values, introducing a simple and effective approach named ReAlign, which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence. This approach minimizes human annotation, hallucination, and the difficulty in scaling, remaining orthogonal to existing alignment techniques. Experimentally, ReAlign significantly boosts the general alignment ability, math reasoning, factuality, and readability of the LLMs. Encouragingly, without introducing any additional data or advanced training techniques, and merely by reformatting the response, LLaMA-2-13B's mathematical reasoning ability on GSM8K can be improved from 46.77% to 56.63% in accuracy. Additionally, a mere 5% of ReAlign data yields a 67% boost in general alignment ability measured by the Alpaca dataset. This work highlights the need for further research into the science and mechanistic interpretability of LLMs. We have made the associated code and data publicly accessible to support future studies at https://github.com/GAIR-NLP/ReAlign.","url_abs":"https://arxiv.org/abs/2402.12219v2","url_pdf":"https://arxiv.org/pdf/2402.12219v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reformatted-alignment","repo_url":"https://github.com/gair-nlp/realign","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.12219","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12219"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gair-nlp/realign","reach":{"status":"ok"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"230c397a7561fedd","entry":"code_selection","repo":"gair-nlp/realign","repo_kind":"official","path":"code/rewrite_data_selection.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/rewrite_data_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"230c397a7561fedd"}},{"code_sha256_prefix":"efed387c112a7d0c","entry":"evidence_str_generate","repo":"gair-nlp/realign","repo_kind":"official","path":"code/reformat.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/reformat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"efed387c112a7d0c"}},{"code_sha256_prefix":"c5c5b7a95ee686a0","entry":"exam_selection","repo":"gair-nlp/realign","repo_kind":"official","path":"code/rewrite_data_selection.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/rewrite_data_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5c5b7a95ee686a0"}},{"code_sha256_prefix":"498b211db19f1ca1","entry":"load_jsonl_data","repo":"gair-nlp/realign","repo_kind":"official","path":"code/util.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"498b211db19f1ca1"}},{"code_sha256_prefix":"42eb4be41a7b16eb","entry":"load_single_json_file","repo":"gair-nlp/realign","repo_kind":"official","path":"code/util.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42eb4be41a7b16eb"}},{"code_sha256_prefix":"260758f1e10222ff","entry":"load_single_jsonl_file","repo":"gair-nlp/realign","repo_kind":"official","path":"code/util.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"260758f1e10222ff"}},{"code_sha256_prefix":"26abe34261698567","entry":"math_selection","repo":"gair-nlp/realign","repo_kind":"official","path":"code/rewrite_data_selection.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/rewrite_data_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26abe34261698567"}},{"code_sha256_prefix":"bb3f89a849434b96","entry":"select_optimal_response","repo":"gair-nlp/realign","repo_kind":"official","path":"code/reformat.py","file_url":"https://github.com/gair-nlp/realign/blob/HEAD/code/reformat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bb3f89a849434b96"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}