{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-03048","title":"The Value Function Semi-Algebraic Set in Partially Observable Markov Decision Processes","arxiv_id":"2606.03048","date":"2026-06-02","proceeding":null,"authors":["Ryan A. Anderson","Guido Montufar"],"abstract":"We study the geometry of feasible value functions in infinite-horizon partially observable Markov decision processes (POMDPs) under memoryless stochastic policies. Our main contribution is a characterization of the feasible set of value functions as a semi-algebraic set, defined by explicit polynomial inequalities determined by the transition dynamics, observation kernel, and reward structure of the POMDP. This result extends prior work for fully observable Markov decision processes, where the feasible set is known to be a polytope, to the substantially more intricate partially observable setting. In contrast to the polyhedral structure arising in MDPs, partial observability induces fundamentally nonlinear constraints, leading to a richer and more complex geometric structure. Our geometric characterization provides new insight into the landscape of policy optimization in both MDPs and POMDPs, and reveals qualitative phenomena unique to partial observability, including the emergence of isolated local maximizers of the long-term reward and their dependence on the initial state distribution.","url_abs":"https://arxiv.org/abs/2606.03048","url_pdf":"https://arxiv.org/pdf/2606.03048","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.03048","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.03048"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ryan-a-anderson/pomdp-value-geometry","reach":null}],"summary":{"ran":9,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"629bb4ddfcab072c","entry":"dirichlet_sample","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_localopt.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_localopt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"629bb4ddfcab072c"}},{"code_sha256_prefix":"cadf730b9f604516","entry":"generate_random_pomdp","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/local_optima_experiments.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/local_optima_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cadf730b9f604516"}},{"code_sha256_prefix":"20878967b64874fe","entry":"plot_multistart_comparison","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/make_figures.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/make_figures.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20878967b64874fe"}},{"code_sha256_prefix":"8472ecf11917d6c5","entry":"policy_spread_mean","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_localopt.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_localopt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8472ecf11917d6c5"}},{"code_sha256_prefix":"118a31d41e38d436","entry":"run_rho_sweep","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_optim_dynamics_very_noisy.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_optim_dynamics_very_noisy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"118a31d41e38d436"}},{"code_sha256_prefix":"3718085f30b4ad4b","entry":"run_single_rho_histories","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_optim_dynamics_very_noisy.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_optim_dynamics_very_noisy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3718085f30b4ad4b"}},{"code_sha256_prefix":"113a0f2ce97567ba","entry":"softmax","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/local_optima_experiments.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/local_optima_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"113a0f2ce97567ba"}},{"code_sha256_prefix":"406d47ca23e7fb6d","entry":"softmax_columns","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_localopt.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_localopt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"406d47ca23e7fb6d"}},{"code_sha256_prefix":"4c1485ade8a44a74","entry":"softmax_columns","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/pomdp_memory_enhancement.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/pomdp_memory_enhancement.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c1485ade8a44a74"}},{"code_sha256_prefix":"4a978fc8f5fe499a","entry":"solve_and_grad","repo":"ryan-a-anderson/pomdp-value-geometry","repo_kind":"found_in_text","path":"src/local_optima_experiments.py","file_url":"https://github.com/ryan-a-anderson/pomdp-value-geometry/blob/HEAD/src/local_optima_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a978fc8f5fe499a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"stat.ML","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}