{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-04737","title":"Rationality Measurement and Theory for Reinforcement Learning Agents","arxiv_id":"2602.04737","date":"2026-02-04","proceeding":"ICML","authors":["Kejiang Qian","Amos Storkey","Fengxiang He"],"abstract":"This paper proposes a suite of rationality measures and associated theory for reinforcement learning agents, a property increasingly critical yet rarely explored. We define an action in deployment to be perfectly rational if it maximises the hidden true value function in the steepest direction. The expected value discrepancy of a policy's actions against their rational counterparts, culminating over the trajectory in deployment, is defined to be expected rational risk; an empirical average version in training is also defined. Their difference, termed as rational risk gap, is decomposed into (1) an extrinsic component caused by environment shifts between training and deployment, and (2) an intrinsic one due to the algorithm's generalisability in a dynamic environment. They are upper bounded by, respectively, (1) the $1$-Wasserstein distance between transition kernels and initial state distributions in training and deployment, and (2) the empirical Rademacher complexity of the value function class. Our theory suggests hypotheses on the benefits from regularisers (including layer normalisation, $\\ell_2$ regularisation, and weight normalisation) and domain randomisation, as well as the harm from environment shifts. Experiments are in full agreement with these hypotheses. The code is available at https://github.com/EVIEHub/Rationality.","url_abs":"https://arxiv.org/abs/2602.04737","url_pdf":"https://arxiv.org/pdf/2602.04737","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.04737","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.04737"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/EVIEHub/Rationality","reach":null}],"summary":{"unverified":10},"by_repo_kind":{"found_in_text":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ba759404b7669eb6","entry":"average_kernels","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/cliffwalking.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/cliffwalking.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba759404b7669eb6"}},{"code_sha256_prefix":"0dda768efc1f1d5f","entry":"finite_horizon_optimal_Qh","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/utils/utils.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0dda768efc1f1d5f"}},{"code_sha256_prefix":"a26dc98b4e0011dd","entry":"get_base_P","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/cliffwalking.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/cliffwalking.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a26dc98b4e0011dd"}},{"code_sha256_prefix":"e913f8d62c7442e2","entry":"get_base_P","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/taxi.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/taxi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e913f8d62c7442e2"}},{"code_sha256_prefix":"fc74ddaa7275bd96","entry":"l2_param_loss","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/regularisers.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/regularisers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc74ddaa7275bd96"}},{"code_sha256_prefix":"1e39668be1d36380","entry":"make_d0_gym_valid_uniform","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/taxi.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/taxi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1e39668be1d36380"}},{"code_sha256_prefix":"a4e329a9b8967732","entry":"slip_kernel","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/cliffwalking.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/cliffwalking.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a4e329a9b8967732"}},{"code_sha256_prefix":"d00ef60e7b2bf645","entry":"slip_kernel","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/env/taxi.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/env/taxi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d00ef60e7b2bf645"}},{"code_sha256_prefix":"9a40c3f0fdd8dad6","entry":"softmax_probs","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/utils/utils.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9a40c3f0fdd8dad6"}},{"code_sha256_prefix":"3933dd3cc3f50b28","entry":"stochastic_policy_from_Qh","repo":"EVIEHub/Rationality","repo_kind":"found_in_text","path":"src/utils/utils.py","file_url":"https://github.com/EVIEHub/Rationality/blob/HEAD/src/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3933dd3cc3f50b28"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}