{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-uncertainty-aware-temporally","title":"Learning Uncertainty-Aware Temporally-Extended Actions","arxiv_id":"2402.05439","date":"2024-02-08","proceeding":null,"authors":["Joongkyu Lee","Seung Joon Park","Yunhao Tang","Min-hwan Oh"],"abstract":"In reinforcement learning, temporal abstraction in the action space, exemplified by action repetition, is a technique to facilitate policy learning through extended actions. However, a primary limitation in previous studies of action repetition is its potential to degrade performance, particularly when sub-optimal actions are repeated. This issue often negates the advantages of action repetition. To address this, we propose a novel algorithm named Uncertainty-aware Temporal Extension (UTE). UTE employs ensemble methods to accurately measure uncertainty during action extension. This feature allows policies to strategically choose between emphasizing exploration or adopting an uncertainty-averse approach, tailored to their specific needs. We demonstrate the effectiveness of UTE through experiments in Gridworld and Atari 2600 environments. Our findings show that UTE outperforms existing action repetition algorithms, effectively mitigating their inherent limitations and significantly enhancing policy learning efficiency.","url_abs":"https://arxiv.org/abs/2402.05439v1","url_pdf":"https://arxiv.org/pdf/2402.05439v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-uncertainty-aware-temporally","repo_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.05439","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.05439"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","reach":null}],"summary":{"ran":4,"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d3418660ae557153","entry":"EnsembleNet","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3418660ae557153"}},{"code_sha256_prefix":"2dc3436c5274ce88","entry":"NoneConcatSkipReplayBuffer","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2dc3436c5274ce88"}},{"code_sha256_prefix":"07baabc5c9d7f523","entry":"Q","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07baabc5c9d7f523"}},{"code_sha256_prefix":"5023894548dbacbb","entry":"ReplayBuffer","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5023894548dbacbb"}},{"code_sha256_prefix":"f9394ba6ed1fa0af","entry":"tt","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9394ba6ed1fa0af"}},{"code_sha256_prefix":"98d44967f2395000","entry":"UTE","repo":"oh-lab/UTE-Uncertainty-aware-Temporal-Extension-","repo_kind":"listed","path":"chain_mdp/agent/temporal_extension.py","file_url":"https://github.com/oh-lab/UTE-Uncertainty-aware-Temporal-Extension-/blob/HEAD/chain_mdp/agent/temporal_extension.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98d44967f2395000"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}