{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aligning-language-models-to-explicitly-handle","title":"Aligning Language Models to Explicitly Handle Ambiguity","arxiv_id":"2404.11972","date":"2024-04-18","proceeding":null,"authors":["Hyuhng Joon Kim","Youna Kim","Cheonbok Park","Junyeob Kim","Choonghyun Park","Kang Min Yoo","Sang-goo Lee","Taeuk Kim"],"abstract":"In interactions between users and language model agents, user utterances frequently exhibit ellipsis (omission of words or phrases) or imprecision (lack of exactness) to prioritize efficiency. This can lead to varying interpretations of the same input based on different assumptions or background knowledge. It is thus crucial for agents to adeptly handle the inherent ambiguity in queries to ensure reliability. However, even state-of-the-art large language models (LLMs) still face challenges in such scenarios, primarily due to the following hurdles: (1) LLMs are not explicitly trained to deal with ambiguous utterances; (2) the degree of ambiguity perceived by the LLMs may vary depending on the possessed knowledge. To address these issues, we propose Alignment with Perceived Ambiguity (APA), a novel pipeline that aligns LLMs to manage ambiguous queries by leveraging their own assessment of ambiguity (i.e., perceived ambiguity). Experimental results on question-answering datasets demonstrate that APA empowers LLMs to explicitly detect and manage ambiguous queries while retaining the ability to answer clear questions. Furthermore, our finding proves that APA excels beyond training with gold-standard labels, especially in out-of-distribution scenarios. The data and code are available at https://github.com/heyjoonkim/APA.","url_abs":"https://arxiv.org/abs/2404.11972v3","url_pdf":"https://arxiv.org/pdf/2404.11972v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aligning-language-models-to-explicitly-handle","repo_url":"https://github.com/heyjoonkim/apa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"apa","method_name":"APA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.11972","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.11972"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/heyjoonkim/apa","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/heyjoonkim/APA","reach":{"status":"ok"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"b63b0d6cea6c5b8b","entry":"analysis","repo":"heyjoonkim/APA","repo_kind":"official","path":"pipeline/evaluation.py","file_url":"https://github.com/heyjoonkim/APA/blob/HEAD/pipeline/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b63b0d6cea6c5b8b"}},{"code_sha256_prefix":"ba0b4d82b2aefc6a","entry":"load_json","repo":"heyjoonkim/APA","repo_kind":"official","path":"utils/data.py","file_url":"https://github.com/heyjoonkim/APA/blob/HEAD/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba0b4d82b2aefc6a"}},{"code_sha256_prefix":"ebb499162645d3e9","entry":"load_pkl","repo":"heyjoonkim/APA","repo_kind":"official","path":"utils/data.py","file_url":"https://github.com/heyjoonkim/APA/blob/HEAD/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ebb499162645d3e9"}},{"code_sha256_prefix":"838e83918b9599d8","entry":"preprocess","repo":"heyjoonkim/APA","repo_kind":"official","path":"pipeline/preprocess.py","file_url":"https://github.com/heyjoonkim/APA/blob/HEAD/pipeline/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"838e83918b9599d8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}