{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-scale-study-of-curiosity-driven","title":"Large-Scale Study of Curiosity-Driven Learning","arxiv_id":"1808.04355","date":"2018-08-13","proceeding":"ICLR 2019 5","authors":["Yuri Burda","Harri Edwards","Deepak Pathak","Amos Storkey","Trevor Darrell","Alexei A. Efros"],"abstract":"Reinforcement learning algorithms rely on carefully engineering environment\nrewards that are extrinsic to the agent. However, annotating each environment\nwith hand-designed, dense rewards is not scalable, motivating the need for\ndeveloping reward functions that are intrinsic to the agent. Curiosity is a\ntype of intrinsic reward function which uses prediction error as reward signal.\nIn this paper: (a) We perform the first large-scale study of purely\ncuriosity-driven learning, i.e. without any extrinsic rewards, across 54\nstandard benchmark environments, including the Atari game suite. Our results\nshow surprisingly good performance, and a high degree of alignment between the\nintrinsic curiosity objective and the hand-designed extrinsic rewards of many\ngame environments. (b) We investigate the effect of using different feature\nspaces for computing prediction error and show that random features are\nsufficient for many popular RL game benchmarks, but learned features appear to\ngeneralize better (e.g. to novel game levels in Super Mario Bros.). (c) We\ndemonstrate limitations of the prediction-based rewards in stochastic setups.\nGame-play videos and code are at\nhttps://pathak22.github.io/large-scale-curiosity/","url_abs":"http://arxiv.org/abs/1808.04355v1","url_pdf":"http://arxiv.org/pdf/1808.04355v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-scale-study-of-curiosity-driven","repo_url":"https://github.com/openai/large-scale-curiosity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"large-scale-study-of-curiosity-driven","repo_url":"https://github.com/SPark9625/Large-Scale-Study-of-Curiosity-Driven-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"large-scale-study-of-curiosity-driven","repo_url":"https://github.com/jcwleo/curiosity-driven-exploration-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"large-scale-study-of-curiosity-driven","repo_url":"https://github.com/self-supervisor/how_to_stay_curious_while_avoiding_noisy_tvs","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"large-scale-study-of-curiosity-driven","repo_url":"https://github.com/vdean/audio-curiosity","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"snes-games","task_name":"SNES Games"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-freeway","task":"Atari Games","dataset":"Atari 2600 Freeway","model":"Intrinsic Reward Agent","rank_in_archive_order":23,"of":59,"metrics":{"Score":"32.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"Intrinsic Reward Agent","rank_in_archive_order":21,"of":53,"metrics":{"Score":"1165.1"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"Intrinsic Reward Agent","rank_in_archive_order":14,"of":50,"metrics":{"Score":"2504.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"Intrinsic Reward Agent","rank_in_archive_order":16,"of":52,"metrics":{"Score":"3036.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"Intrinsic Reward Agent","rank_in_archive_order":24,"of":55,"metrics":{"Score":"416"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.04355","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1808.04355"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SPark9625/Large-Scale-Study-of-Curiosity-Driven-Learning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vdean/audio-curiosity","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jcwleo/curiosity-driven-exploration-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/self-supervisor/how_to_stay_curious_while_avoiding_noisy_tvs","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openai/large-scale-curiosity","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"9228dfba9596d01f","entry":"assign_color","repo":"vdean/audio-curiosity","repo_kind":"listed","path":"make_plots.py","file_url":"https://github.com/vdean/audio-curiosity/blob/HEAD/make_plots.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9228dfba9596d01f"}},{"code_sha256_prefix":"ae0035ac55ff4a0a","entry":"assign_label","repo":"vdean/audio-curiosity","repo_kind":"listed","path":"make_plots.py","file_url":"https://github.com/vdean/audio-curiosity/blob/HEAD/make_plots.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae0035ac55ff4a0a"}},{"code_sha256_prefix":"2cd075f28d3a33a3","entry":"get_paths","repo":"vdean/audio-curiosity","repo_kind":"listed","path":"make_plots.py","file_url":"https://github.com/vdean/audio-curiosity/blob/HEAD/make_plots.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2cd075f28d3a33a3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}