{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-supervised-exploration-via-latent","title":"Curiosity-Driven Exploration via Latent Bayesian Surprise","arxiv_id":"2104.07495","date":"2021-04-15","proceeding":"ICLR Workshop SSL-RL 2021 5","authors":["Pietro Mazzaglia","Ozan Catal","Tim Verbelen","Bart Dhoedt"],"abstract":"The human intrinsic desire to pursue knowledge, also known as curiosity, is considered essential in the process of skill acquisition. With the aid of artificial curiosity, we could equip current techniques for control, such as Reinforcement Learning, with more natural exploration capabilities. A promising approach in this respect has consisted of using Bayesian surprise on model parameters, i.e. a metric for the difference between prior and posterior beliefs, to favour exploration. In this contribution, we propose to apply Bayesian surprise in a latent space representing the agent's current understanding of the dynamics of the system, drastically reducing the computational costs. We extensively evaluate our method by measuring the agent's performance in terms of environment exploration, for continuous tasks, and looking at the game scores achieved, for video games. Our model is computationally cheap and compares positively with current state-of-the-art methods on several problems. We also investigate the effects caused by stochasticity in the environment, which is often a failure case for curiosity-driven agents. In this regime, the results suggest that our approach is resilient to stochastic transitions.","url_abs":"https://arxiv.org/abs/2104.07495v2","url_pdf":"https://arxiv.org/pdf/2104.07495v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-supervised-exploration-via-latent","repo_url":"https://github.com/mazpie/lbs-exploration","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"self-supervised-exploration-via-latent","repo_url":"https://github.com/mazpie/mastering-urlb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.07495","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.07495"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mazpie/mastering-urlb","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mazpie/lbs-exploration","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":8},"by_repo_kind":{"official":{"samples":10,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9de355e93051e4ad","entry":"init","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/utils.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9de355e93051e4ad"}},{"code_sha256_prefix":"f2e3732f89410bea","entry":"unwrap","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/envs.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/envs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2e3732f89410bea"}},{"code_sha256_prefix":"35be8e93eeee05c1","entry":"get_conv_net","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/models/common.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35be8e93eeee05c1"}},{"code_sha256_prefix":"ac436b1c63a904a4","entry":"get_random_cnn","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/models/common.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac436b1c63a904a4"}},{"code_sha256_prefix":"761a664f42bbe01d","entry":"get_random_mlp","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/models/common.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/models/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"761a664f42bbe01d"}},{"code_sha256_prefix":"108dc9c41448855e","entry":"get_render_func","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/utils.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"108dc9c41448855e"}},{"code_sha256_prefix":"e426cc24815ec216","entry":"get_state_block","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/environments/half_cheetah_sparse.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/environments/half_cheetah_sparse.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e426cc24815ec216"}},{"code_sha256_prefix":"61ad77d7ae8949a8","entry":"get_state_block","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/environments/magellan_ant.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/environments/magellan_ant.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61ad77d7ae8949a8"}},{"code_sha256_prefix":"6f9031055aa08aed","entry":"rate_buffer","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/environments/magellan_ant.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/environments/magellan_ant.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f9031055aa08aed"}},{"code_sha256_prefix":"f9bbb8bf4db866b6","entry":"rate_buffer_with_blocks","repo":"mazpie/lbs-exploration","repo_kind":"official","path":"exploration/environments/half_cheetah_sparse.py","file_url":"https://github.com/mazpie/lbs-exploration/blob/HEAD/exploration/environments/half_cheetah_sparse.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bbb8bf4db866b6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}