{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eigenoption-discovery-through-the-deep","title":"Eigenoption Discovery through the Deep Successor Representation","arxiv_id":"1710.11089","date":"2017-10-30","proceeding":"ICLR 2018 1","authors":["Marlos C. Machado","Clemens Rosenbaum","Xiaoxiao Guo","Miao Liu","Gerald Tesauro","Murray Campbell"],"abstract":"Options in reinforcement learning allow agents to hierarchically decompose a\ntask into subtasks, having the potential to speed up learning and planning.\nHowever, autonomously learning effective sets of options is still a major\nchallenge in the field. In this paper we focus on the recently introduced idea\nof using representation learning methods to guide the option discovery process.\nSpecifically, we look at eigenoptions, options obtained from representations\nthat encode diffusive information flow in the environment. We extend the\nexisting algorithms for eigenoption discovery to settings with stochastic\ntransitions and in which handcrafted features are not available. We propose an\nalgorithm that discovers eigenoptions while learning non-linear state\nrepresentations from raw pixels. It exploits recent successes in the deep\nreinforcement learning literature and the equivalence between proto-value\nfunctions and the successor representation. We use traditional tabular domains\nto provide intuition about our approach and Atari 2600 games to demonstrate its\npotential.","url_abs":"http://arxiv.org/abs/1710.11089v3","url_pdf":"http://arxiv.org/pdf/1710.11089v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eigenoption-discovery-through-the-deep","repo_url":"https://github.com/mklissa/dceo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1710.11089","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1710.11089"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mklissa/dceo","reach":null}],"summary":{"ran_fixture":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bbebc7418e5cfd5c","entry":"act","repo":"mklissa/dceo","repo_kind":"listed","path":"dopamine/jax/agents/full_rainbow/full_rainbow_dceo.py","file_url":"https://github.com/mklissa/dceo/blob/HEAD/dopamine/jax/agents/full_rainbow/full_rainbow_dceo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bbebc7418e5cfd5c"}},{"code_sha256_prefix":"2957ca0c26f2fc3f","entry":"select_action","repo":"mklissa/dceo","repo_kind":"listed","path":"dopamine/jax/agents/full_rainbow/full_rainbow_dceo.py","file_url":"https://github.com/mklissa/dceo/blob/HEAD/dopamine/jax/agents/full_rainbow/full_rainbow_dceo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2957ca0c26f2fc3f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}