{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/alignment-for-honesty","title":"Alignment for Honesty","arxiv_id":"2312.07000","date":"2023-12-12","proceeding":null,"authors":["Yuqing Yang","Ethan Chern","Xipeng Qiu","Graham Neubig","PengFei Liu"],"abstract":"Recent research has made significant strides in aligning large language models (LLMs) with helpfulness and harmlessness. In this paper, we argue for the importance of alignment for \\emph{honesty}, ensuring that LLMs proactively refuse to answer questions when they lack knowledge, while still not being overly conservative. However, a pivotal aspect of alignment for honesty involves discerning an LLM's knowledge boundaries, which demands comprehensive solutions in terms of metric development, benchmark creation, and training methodologies. We address these challenges by first establishing a precise problem definition and defining ``honesty'' inspired by the Analects of Confucius. This serves as a cornerstone for developing metrics that effectively measure an LLM's honesty by quantifying its progress post-alignment. Furthermore, we introduce a flexible training framework which is further instantiated by several efficient fine-tuning techniques that emphasize honesty without sacrificing performance on other tasks. Our extensive experiments reveal that these aligned models show a marked increase in honesty, as indicated by our proposed metrics. We open-source all relevant resources to facilitate future research at \\url{https://github.com/GAIR-NLP/alignment-for-honesty}.","url_abs":"https://arxiv.org/abs/2312.07000v2","url_pdf":"https://arxiv.org/pdf/2312.07000v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"alignment-for-honesty","repo_url":"https://github.com/gair-nlp/alignment-for-honesty","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.07000","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.07000"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/GAIR-NLP/alignment-for-honesty","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gair-nlp/alignment-for-honesty","reach":{"status":"ok"}}],"summary":{"ran":9,"unverified":2},"by_repo_kind":{"official":{"samples":11,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"647d5f0831f57b1c","entry":"calculate_f1","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"647d5f0831f57b1c"}},{"code_sha256_prefix":"6f9f1cd8cfc72e62","entry":"correct_by_chatgpt_score","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f9f1cd8cfc72e62"}},{"code_sha256_prefix":"81413be8207e7982","entry":"has_exact_match_score","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/triviaqa_nonambigqa_evaluation.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/triviaqa_nonambigqa_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"81413be8207e7982"}},{"code_sha256_prefix":"58218212a8e56872","entry":"heuristic_idk","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"58218212a8e56872"}},{"code_sha256_prefix":"f9b2d6416d25bd70","entry":"is_exact_match_score","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/triviaqa_nonambigqa_evaluation.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/triviaqa_nonambigqa_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9b2d6416d25bd70"}},{"code_sha256_prefix":"c29fc928dae85749","entry":"normalize_answer","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/triviaqa_nonambigqa_evaluation.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/triviaqa_nonambigqa_evaluation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c29fc928dae85749"}},{"code_sha256_prefix":"09750e72b3959dec","entry":"process_data","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/pkqa_evaluation.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/pkqa_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09750e72b3959dec"}},{"code_sha256_prefix":"be39151de2fd6637","entry":"process_data1","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/triviaqa_nonambigqa_chatgpt.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/triviaqa_nonambigqa_chatgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"be39151de2fd6637"}},{"code_sha256_prefix":"954e91478074018d","entry":"process_data2","repo":"GAIR-NLP/alignment-for-honesty","repo_kind":"official","path":"evaluation/triviaqa_nonambigqa_chatgpt.py","file_url":"https://github.com/GAIR-NLP/alignment-for-honesty/blob/HEAD/evaluation/triviaqa_nonambigqa_chatgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"954e91478074018d"}},{"code_sha256_prefix":"4e2e8c1710835755","entry":"SFTMethod","repo":"gair-nlp/alignment-for-honesty","repo_kind":"official","path":"train/process_training_data.py","file_url":"https://github.com/gair-nlp/alignment-for-honesty/blob/HEAD/train/process_training_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e2e8c1710835755"}},{"code_sha256_prefix":"0d8982e79978c4cc","entry":"process_data","repo":"gair-nlp/alignment-for-honesty","repo_kind":"official","path":"evaluation/mmlu_evaluation.py","file_url":"https://github.com/gair-nlp/alignment-for-honesty/blob/HEAD/evaluation/mmlu_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d8982e79978c4cc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}