{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/predicting-future-actions-of-reinforcement","title":"Predicting Future Actions of Reinforcement Learning Agents","arxiv_id":"2410.22459","date":"2024-10-29","proceeding":null,"authors":["Stephen Chung","Scott Niekum","David Krueger"],"abstract":"As reinforcement learning agents become increasingly deployed in real-world scenarios, predicting future agent actions and events during deployment is important for facilitating better human-agent interaction and preventing catastrophic outcomes. This paper experimentally evaluates and compares the effectiveness of future action and event prediction for three types of RL agents: explicitly planning, implicitly planning, and non-planning. We employ two approaches: the inner state approach, which involves predicting based on the inner computations of the agents (e.g., plans or neuron activations), and a simulation-based approach, which involves unrolling the agent in a learned world model. Our results show that the plans of explicitly planning agents are significantly more informative for prediction than the neuron activations of the other types. Furthermore, using internal plans proves more robust to model quality compared to simulation-based approaches when predicting actions, while the results for event prediction are more mixed. These findings highlight the benefits of leveraging inner states and simulations to predict future agent actions and events, thereby improving interaction and safety in real-world deployments.","url_abs":"https://arxiv.org/abs/2410.22459v1","url_pdf":"https://arxiv.org/pdf/2410.22459v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"predicting-future-actions-of-reinforcement","repo_url":"https://github.com/stephen-chung-mh/predict_action","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.22459","atlas_url":"https://app.syntology.ai/?focus=2410.22459","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.22459"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stephen-chung-mh/predict_action","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0ef3a5ae92f974da","entry":"conv_greyscale","repo":"stephen-chung-mh/predict_action","repo_kind":"official","path":"thinker/detect/detect_gen.py","file_url":"https://github.com/stephen-chung-mh/predict_action/blob/HEAD/thinker/detect/detect_gen.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ef3a5ae92f974da"}},{"code_sha256_prefix":"056907c66b08dba8","entry":"evaluate_detect","repo":"stephen-chung-mh/predict_action","repo_kind":"official","path":"thinker/detect/detect_train.py","file_url":"https://github.com/stephen-chung-mh/predict_action/blob/HEAD/thinker/detect/detect_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"056907c66b08dba8"}},{"code_sha256_prefix":"ef34b3bf8afad266","entry":"transform_data","repo":"stephen-chung-mh/predict_action","repo_kind":"official","path":"thinker/detect/detect_train.py","file_url":"https://github.com/stephen-chung-mh/predict_action/blob/HEAD/thinker/detect/detect_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef34b3bf8afad266"}},{"code_sha256_prefix":"b275dd366665360d","entry":"train_epoch","repo":"stephen-chung-mh/predict_action","repo_kind":"official","path":"thinker/detect/detect_train.py","file_url":"https://github.com/stephen-chung-mh/predict_action/blob/HEAD/thinker/detect/detect_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b275dd366665360d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}