{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-laplacian-framework-for-option-discovery-in","title":"A Laplacian Framework for Option Discovery in Reinforcement Learning","arxiv_id":"1703.00956","date":"2017-03-02","proceeding":"ICML 2017 8","authors":["Marlos C. Machado","Marc G. Bellemare","Michael Bowling"],"abstract":"Representation learning and option discovery are two of the biggest\nchallenges in reinforcement learning (RL). Proto-value functions (PVFs) are a\nwell-known approach for representation learning in MDPs. In this paper we\naddress the option discovery problem by showing how PVFs implicitly define\noptions. We do it by introducing eigenpurposes, intrinsic reward functions\nderived from the learned representations. The options discovered from\neigenpurposes traverse the principal directions of the state space. They are\nuseful for multiple tasks because they are discovered without taking the\nenvironment's rewards into consideration. Moreover, different options act at\ndifferent time scales, making them helpful for exploration. We demonstrate\nfeatures of eigenpurposes in traditional tabular domains as well as in Atari\n2600 games.","url_abs":"http://arxiv.org/abs/1703.00956v2","url_pdf":"http://arxiv.org/pdf/1703.00956v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-laplacian-framework-for-option-discovery-in","repo_url":"https://github.com/mcmachado/options","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.00956","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}