{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-through-random-curiosity-with","title":"Exploring through Random Curiosity with General Value Functions","arxiv_id":"2211.10282","date":"2022-11-18","proceeding":null,"authors":["Aditya Ramesh","Louis Kirsch","Sjoerd van Steenkiste","Jürgen Schmidhuber"],"abstract":"Efficient exploration in reinforcement learning is a challenging problem commonly addressed through intrinsic rewards. Recent prominent approaches are based on state novelty or variants of artificial curiosity. However, directly applying them to partially observable environments can be ineffective and lead to premature dissipation of intrinsic rewards. Here we propose random curiosity with general value functions (RC-GVF), a novel intrinsic reward function that draws upon connections between these distinct approaches. Instead of using only the current observation's novelty or a curiosity bonus for failing to predict precise environment dynamics, RC-GVF derives intrinsic rewards through predicting temporally extended general value functions. We demonstrate that this improves exploration in a hard-exploration diabolical lock problem. Furthermore, RC-GVF significantly outperforms previous methods in the absence of ground-truth episodic counts in the partially observable MiniGrid environments. Panoramic observations on MiniGrid further boost RC-GVF's performance such that it is competitive to baselines exploiting privileged information in form of episodic counts.","url_abs":"https://arxiv.org/abs/2211.10282v1","url_pdf":"https://arxiv.org/pdf/2211.10282v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-through-random-curiosity-with","repo_url":"https://github.com/aditya-ramesh-10/exploring-through-rcgvf","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"efficient-exploration","task_name":"Efficient Exploration"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.10282","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.10282"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aditya-ramesh-10/exploring-through-rcgvf","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4e866287a3a17c26","entry":"is_agent_in_room","repo":"aditya-ramesh-10/exploring-through-rcgvf","repo_kind":"official","path":"mgrid_utils/env.py","file_url":"https://github.com/aditya-ramesh-10/exploring-through-rcgvf/blob/HEAD/mgrid_utils/env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e866287a3a17c26"}},{"code_sha256_prefix":"2f470de73c0eda3c","entry":"make_env","repo":"aditya-ramesh-10/exploring-through-rcgvf","repo_kind":"official","path":"mgrid_utils/env.py","file_url":"https://github.com/aditya-ramesh-10/exploring-through-rcgvf/blob/HEAD/mgrid_utils/env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f470de73c0eda3c"}},{"code_sha256_prefix":"aae66f156a5e4829","entry":"preprocess_images","repo":"aditya-ramesh-10/exploring-through-rcgvf","repo_kind":"official","path":"mgrid_utils/format.py","file_url":"https://github.com/aditya-ramesh-10/exploring-through-rcgvf/blob/HEAD/mgrid_utils/format.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aae66f156a5e4829"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}