{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uncertainty-of-thoughts-uncertainty-aware","title":"Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models","arxiv_id":"2402.03271","date":"2024-02-05","proceeding":null,"authors":["Zhiyuan Hu","Chumin Liu","Xidong Feng","Yilun Zhao","See-Kiong Ng","Anh Tuan Luu","Junxian He","Pang Wei Koh","Bryan Hooi"],"abstract":"In the face of uncertainty, the ability to *seek information* is of fundamental importance. In many practical applications, such as medical diagnosis and troubleshooting, the information needed to solve the task is not initially given and has to be actively sought by asking follow-up questions (for example, a doctor asking a patient for more details about their symptoms). In this work, we introduce Uncertainty of Thoughts (UoT), an algorithm to augment large language models with the ability to actively seek information by asking effective questions. UoT combines 1) an *uncertainty-aware simulation approach* which enables the model to simulate possible future scenarios and how likely they are to occur, 2) *uncertainty-based rewards* motivated by information gain which incentivizes the model to seek information, and 3) a *reward propagation scheme* to select the optimal question to ask in a way that maximizes the expected reward. In experiments on medical diagnosis, troubleshooting, and the `20 Questions` game, UoT achieves an average performance improvement of 38.1% in the rate of successful task completion across multiple LLMs compared with direct prompting and also improves efficiency (i.e., the number of questions needed to complete the task). Our code has been released [here](https://github.com/zhiyuanhubj/UoT)","url_abs":"https://arxiv.org/abs/2402.03271v3","url_pdf":"https://arxiv.org/pdf/2402.03271v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uncertainty-of-thoughts-uncertainty-aware","repo_url":"https://github.com/zhiyuanhubj/uot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"medical-diagnosis","task_name":"Medical Diagnosis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.03271","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03271"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhiyuanhubj/uot","reach":null}],"summary":{"ran_draft_wrong":2,"ran_fixture":1,"unverified":13},"by_repo_kind":{"official":{"samples":16,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"04d1a1be5ddfeabb","entry":"expand","repo":"zhiyuanhubj/UoT","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/UoT/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04d1a1be5ddfeabb"}},{"code_sha256_prefix":"78ea94ba98dd5c7e","entry":"get_response_method","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78ea94ba98dd5c7e"}},{"code_sha256_prefix":"9e57c988cd778e2c","entry":"select","repo":"zhiyuanhubj/UoT","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/UoT/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e57c988cd778e2c"}},{"code_sha256_prefix":"faacd92c97972182","entry":"UoTNode","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"faacd92c97972182"}},{"code_sha256_prefix":"f235cb46e18a2b2e","entry":"claude_aiproxy_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f235cb46e18a2b2e"}},{"code_sha256_prefix":"c0bf5fb6eb29a6da","entry":"claude_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0bf5fb6eb29a6da"}},{"code_sha256_prefix":"1431d54337e709c6","entry":"cls_given_repo","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1431d54337e709c6"}},{"code_sha256_prefix":"dbe4ff130d5e3c92","entry":"cohere_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbe4ff130d5e3c92"}},{"code_sha256_prefix":"d1b5984ceb754e81","entry":"gemini_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d1b5984ceb754e81"}},{"code_sha256_prefix":"70e3588a0feeed10","entry":"gemma_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70e3588a0feeed10"}},{"code_sha256_prefix":"86a385f2ad6d6e4a","entry":"gpt_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86a385f2ad6d6e4a"}},{"code_sha256_prefix":"6bc34a32108b0301","entry":"initialize_open_set","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bc34a32108b0301"}},{"code_sha256_prefix":"da21a1ecdc43f3f8","entry":"llama_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da21a1ecdc43f3f8"}},{"code_sha256_prefix":"400e7419d1f033b5","entry":"mistral_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"400e7419d1f033b5"}},{"code_sha256_prefix":"3d08c4e8f0220f4e","entry":"palm_response","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d08c4e8f0220f4e"}},{"code_sha256_prefix":"3cffba530cc1ff45","entry":"ques_and_cls_given_items","repo":"zhiyuanhubj/uot","repo_kind":"official","path":"src/uot/uot.py","file_url":"https://github.com/zhiyuanhubj/uot/blob/HEAD/src/uot/uot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3cffba530cc1ff45"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}