{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maximum-entropy-gain-exploration-for-long","title":"Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning","arxiv_id":"2007.02832","date":"2020-07-06","proceeding":"ICML 2020 1","authors":["Silviu Pitis","Harris Chan","Stephen Zhao","Bradly Stadie","Jimmy Ba"],"abstract":"What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a useful learning signal, we argue that the agent should not pursue unobtainable goals. Instead, it should set its own intrinsic goals that maximize the entropy of the historical achieved goal distribution. We propose to optimize this objective by having the agent pursue past achieved goals in sparsely explored areas of the goal space, which focuses exploration on the frontier of the achievable goal set. We show that our strategy achieves an order of magnitude better sample efficiency than the prior state of the art on long-horizon multi-goal tasks including maze navigation and block stacking.","url_abs":"https://arxiv.org/abs/2007.02832v1","url_pdf":"https://arxiv.org/pdf/2007.02832v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"maximum-entropy-gain-exploration-for-long","repo_url":"https://github.com/spitis/mrl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"maximum-entropy-gain-exploration-for-long","repo_url":"https://github.com/penn-pal-lab/peg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"multi-goal-reinforcement-learning","task_name":"Multi-Goal Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.02832","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.02832"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/spitis/mrl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/penn-pal-lab/peg","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3847351048947c67","entry":"MEGA","repo":"penn-pal-lab/peg","repo_kind":"listed","path":"dreamerv2/goal_picker.py","file_url":"https://github.com/penn-pal-lab/peg/blob/HEAD/dreamerv2/goal_picker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3847351048947c67"}},{"code_sha256_prefix":"083546dc5ac62494","entry":"softmax","repo":"penn-pal-lab/peg","repo_kind":"listed","path":"dreamerv2/goal_picker.py","file_url":"https://github.com/penn-pal-lab/peg/blob/HEAD/dreamerv2/goal_picker.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"083546dc5ac62494"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}