{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-09988","title":"OpenPM: Auditable Point-in-Time Evaluation for LLM Portfolio-Management Agents","arxiv_id":"2608.09988","date":"2026-08-06","proceeding":null,"authors":["Xinying Cai","Minghao Guo","Jiahe Liu","Jiaojiao Han","Bangwei Guo","Yitao Long","Yuxuan Chen","Bohan Wu","Dimitris N. Metaxas","Raymond Li"],"abstract":"Large language models are increasingly used to read markets, assess risk, and allocate capital. However, reported results for LLM trading agents can be inflated by look-ahead leakage, optimistic execution, and risk mandates that are described but not enforced. We present OpenPM, an auditable point-in-time evaluation framework for LLM portfolio-management agents. In OpenPM, an agent manages a \\$1M long-only book over the S\\&P 500 universe using market data at five-minute intervals. Every record visible to the agent must be available at the decision time. Natural-language risk mandates are converted into typed constraints and enforced on the executed portfolio. Each run produces audit artifacts, including a contamination certificate, a cost-sensitivity curve, and a constraint-adherence report. We also build a reference agent named the tiered allocator, where typed analysts score candidates, a constructor LLM proposes weights, and a deterministic critic guarantees feasibility. We isolate constructor behavior by capturing analyst evidence once and replaying it across constructor models. In our short-window case study, stronger constructors show modest and model-dependent gains over equal weighting on the same pool, but analyst quality matters more than constructor choice, and turnover is the main cost driver. All returns are upper bounds on a single frozen window without market impact, not validated alpha.","url_abs":"https://arxiv.org/abs/2608.09988","url_pdf":"https://arxiv.org/pdf/2608.09988","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2608.09988","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.09988"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/aslcai/OpenPM-Bench","reach":null}],"summary":{"ran":13,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":16,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"997de451cd98526f","entry":"check_bad_prices","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/data_quality.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/data_quality.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"997de451cd98526f"}},{"code_sha256_prefix":"fadfe44e7611fd92","entry":"check_missing_bars","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/data_quality.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/data_quality.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fadfe44e7611fd92"}},{"code_sha256_prefix":"b25115fe9bc4a86e","entry":"check_zero_volume","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/data_quality.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/data_quality.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b25115fe9bc4a86e"}},{"code_sha256_prefix":"548e1e5b0ad7978b","entry":"count_rows","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/freeze.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/freeze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"548e1e5b0ad7978b"}},{"code_sha256_prefix":"909aae065d34d8ae","entry":"feature_output_filename","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature_paths.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature_paths.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"909aae065d34d8ae"}},{"code_sha256_prefix":"03c77445ee83ba60","entry":"get_sp500_members_between","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/universe.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/universe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03c77445ee83ba60"}},{"code_sha256_prefix":"5cd03955f627a946","entry":"get_sp500_members_on","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/universe.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/universe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5cd03955f627a946"}},{"code_sha256_prefix":"1d1acb98f1c8dc2e","entry":"iter_files","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/freeze.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/freeze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d1acb98f1c8dc2e"}},{"code_sha256_prefix":"22bc40ed82cfcf28","entry":"resolve_symbols_filter","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"22bc40ed82cfcf28"}},{"code_sha256_prefix":"8862dc4ae1fa2b0f","entry":"sha256_file","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/freeze.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/freeze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8862dc4ae1fa2b0f"}},{"code_sha256_prefix":"c03077de44bf2a39","entry":"summarize","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature_validation.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature_validation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c03077de44bf2a39"}},{"code_sha256_prefix":"087ee2f93308c3f1","entry":"symbol_scoped_feature_output","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature_paths.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature_paths.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"087ee2f93308c3f1"}},{"code_sha256_prefix":"8cbc747c123c40b3","entry":"validate_rows","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature_validation.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature_validation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8cbc747c123c40b3"}},{"code_sha256_prefix":"e5e4873ca3a5df53","entry":"cross_sections_from_panel","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/portfolio/backtester.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/portfolio/backtester.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5e4873ca3a5df53"}},{"code_sha256_prefix":"9444feff60cdff1c","entry":"get_sp500_member_rows_on","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"collectors/universe.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/collectors/universe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9444feff60cdff1c"}},{"code_sha256_prefix":"30f4a3867b83de47","entry":"validate_file","repo":"aslcai/OpenPM-Bench","repo_kind":"found_in_text","path":"agents/feature_validation.py","file_url":"https://github.com/aslcai/OpenPM-Bench/blob/HEAD/agents/feature_validation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"30f4a3867b83de47"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}