{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-13740","title":"Learning POMDP World Models from Observations with Language-Model Priors","arxiv_id":"2605.13740","date":"2026-05-13","proceeding":null,"authors":["Valentin Six","Frederik Panse","Mathis Fajeau","Lancelot Da Costa","Mridul Sharma","Alfonso Amayuelas","Tim Z. Xiao","David Hyland","Philipp Hennig","Bernhard Schölkopf"],"abstract":"Whether navigating a building, operating a robot, or playing a game, an agent that acts effectively in an environment must first learn an internal model of how that environment works. Partially-observable Markov decision processes (POMDPs) provide a flexible modeling class for such internal world models, but learning them from observation-action trajectories alone is challenging and typically requires extensive environment interaction. We ask whether language-model priors can reduce costly interaction by leveraging prior knowledge, and introduce \\emph{Pinductor} (POMDP-inductor): an LLM proposes candidate POMDP models from a few observation-action trajectories and iteratively refines them to optimize a belief-based likelihood score. Despite using strictly less information, \\emph{Pinductor} matches the performance and sample efficiency of LLM-based POMDP learning methods that assume privileged access to the hidden state, while significantly surpassing the sample efficiency of tabular POMDP baselines. Further results show that performance scales with LLM capability and degrades gracefully as semantic information about the environment is withheld. Together, these results position language-model priors as a practical tool for sample-efficient world-model learning under partial observability, and a step toward generalist agents in real-world environments. Code is available at https://github.com/atomresearch/pinductor.","url_abs":"https://arxiv.org/abs/2605.13740","url_pdf":"https://arxiv.org/pdf/2605.13740","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.13740"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/atomresearch/pinductor","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/aidan-curtis/pomdp_coder","reach":null}],"summary":{"ran":9,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":14,"ran":9,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"0ed9d1fd098bfc46","entry":"discounted_reward","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ed9d1fd098bfc46"}},{"code_sha256_prefix":"9b799aed2b6476c1","entry":"discounted_reward","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b799aed2b6476c1"}},{"code_sha256_prefix":"d8607781508bded2","entry":"normalize","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/planners/PO_DAStar.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/planners/PO_DAStar.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8607781508bded2"}},{"code_sha256_prefix":"3f2596ece2e1bf5c","entry":"parse_prompt","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f2596ece2e1bf5c"}},{"code_sha256_prefix":"5b60c9c7b3041e0b","entry":"reward_MSE","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/metrics.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b60c9c7b3041e0b"}},{"code_sha256_prefix":"8ada5631fbacf671","entry":"rollout_fn","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/planners/FO_DAStar.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/planners/FO_DAStar.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8ada5631fbacf671"}},{"code_sha256_prefix":"1dc655dd260e8cab","entry":"rollout_fn","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/planners/PO_DAStar.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/planners/PO_DAStar.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1dc655dd260e8cab"}},{"code_sha256_prefix":"5d8892ebe44cbf14","entry":"score_trajectory","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"particle_filtering/belief_quality_scorer.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/particle_filtering/belief_quality_scorer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d8892ebe44cbf14"}},{"code_sha256_prefix":"e39125d4828ba6bc","entry":"term_cross_entropy","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/policies/llm_feedback.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/policies/llm_feedback.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e39125d4828ba6bc"}},{"code_sha256_prefix":"7b38e4ef4bbf36ee","entry":"likelihood_components_from_episode_metrics","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/policies/llm_feedback.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/policies/llm_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b38e4ef4bbf36ee"}},{"code_sha256_prefix":"5e6fe57b80480c02","entry":"likelihood_from_episode_metrics","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/policies/llm_feedback.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/policies/llm_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e6fe57b80480c02"}},{"code_sha256_prefix":"5b5bc89673a1f23e","entry":"parse_code","repo":"aidan-curtis/pomdp_coder","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/aidan-curtis/pomdp_coder/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b5bc89673a1f23e"}},{"code_sha256_prefix":"17fae40e584583c4","entry":"parse_code","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17fae40e584583c4"}},{"code_sha256_prefix":"25c69312cf46044a","entry":"parse_prompt","repo":"atomresearch/pinductor","repo_kind":"found_in_text","path":"uncertain_worms/utils.py","file_url":"https://github.com/atomresearch/pinductor/blob/HEAD/uncertain_worms/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25c69312cf46044a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}