{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-elicitation-of-latent-information","title":"Adaptive Elicitation of Latent Information Using Natural Language","arxiv_id":"2504.04204","date":"2025-04-05","proceeding":null,"authors":["Jimmy Wang","Thomas Zollo","Richard Zemel","Hongseok Namkoong"],"abstract":"Eliciting information to reduce uncertainty about a latent entity is a critical task in many application domains, e.g., assessing individual student learning outcomes, diagnosing underlying diseases, or learning user preferences. Though natural language is a powerful medium for this purpose, large language models (LLMs) and existing fine-tuning algorithms lack mechanisms for strategically gathering information to refine their own understanding of the latent entity. To harness the generalization power and world knowledge of LLMs in developing effective information-gathering strategies, we propose an adaptive elicitation framework that actively reduces uncertainty on the latent entity. Since probabilistic modeling of an abstract latent entity is difficult, our framework adopts a predictive view of uncertainty, using a meta-learned language model to simulate future observations and enable scalable uncertainty quantification over complex natural language. Through autoregressive forward simulation, our model quantifies how new questions reduce epistemic uncertainty, enabling the development of sophisticated information-gathering strategies to choose the most informative next queries. In experiments on the 20 questions game, dynamic opinion polling, and adaptive student assessment, our method consistently outperforms baselines in identifying critical unknowns and improving downstream predictions, illustrating the promise of strategic information gathering in natural language settings.","url_abs":"https://arxiv.org/abs/2504.04204v1","url_pdf":"https://arxiv.org/pdf/2504.04204v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.04204","atlas_url":"https://app.syntology.ai/?focus=2504.04204","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.04204"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/namkoong-lab/adaptive-elicitation","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a52ad800f5551178","entry":"get_corresponding_logits","repo":"namkoong-lab/adaptive-elicitation","repo_kind":"found_in_text","path":"src/eval/info_gain.py","file_url":"https://github.com/namkoong-lab/adaptive-elicitation/blob/HEAD/src/eval/info_gain.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a52ad800f5551178"}},{"code_sha256_prefix":"cdafbd2b43382e5c","entry":"get_target_condition_probs","repo":"namkoong-lab/adaptive-elicitation","repo_kind":"found_in_text","path":"src/eval/info_gain.py","file_url":"https://github.com/namkoong-lab/adaptive-elicitation/blob/HEAD/src/eval/info_gain.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdafbd2b43382e5c"}},{"code_sha256_prefix":"5113eb5b7fffa7ce","entry":"MCTS","repo":"namkoong-lab/adaptive-elicitation","repo_kind":"found_in_text","path":"src/eval/info_gain.py","file_url":"https://github.com/namkoong-lab/adaptive-elicitation/blob/HEAD/src/eval/info_gain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5113eb5b7fffa7ce"}},{"code_sha256_prefix":"cce3b71b69c1daae","entry":"calc_info_gain","repo":"namkoong-lab/adaptive-elicitation","repo_kind":"found_in_text","path":"src/eval/info_gain.py","file_url":"https://github.com/namkoong-lab/adaptive-elicitation/blob/HEAD/src/eval/info_gain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cce3b71b69c1daae"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}