{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stochastic-neural-networks-for-hierarchical","title":"Stochastic Neural Networks for Hierarchical Reinforcement Learning","arxiv_id":"1704.03012","date":"2017-04-10","proceeding":null,"authors":["Carlos Florensa","Yan Duan","Pieter Abbeel"],"abstract":"Deep reinforcement learning has achieved many impressive results in recent\nyears. However, tasks with sparse rewards or long horizons continue to pose\nsignificant challenges. To tackle these important problems, we propose a\ngeneral framework that first learns useful skills in a pre-training\nenvironment, and then leverages the acquired skills for learning faster in\ndownstream tasks. Our approach brings together some of the strengths of\nintrinsic motivation and hierarchical methods: the learning of useful skill is\nguided by a single proxy reward, the design of which requires very minimal\ndomain knowledge about the downstream tasks. Then a high-level policy is\ntrained on top of these skills, providing a significant improvement of the\nexploration and allowing to tackle sparse rewards in the downstream tasks. To\nefficiently pre-train a large span of skills, we use Stochastic Neural Networks\ncombined with an information-theoretic regularizer. Our experiments show that\nthis combination is effective in learning a wide span of interpretable skills\nin a sample-efficient way, and can significantly boost the learning performance\nuniformly across a wide range of downstream tasks.","url_abs":"http://arxiv.org/abs/1704.03012v1","url_pdf":"http://arxiv.org/pdf/1704.03012v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stochastic-neural-networks-for-hierarchical","repo_url":"https://github.com/florensacc/snn4hrl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"stochastic-neural-networks-for-hierarchical","repo_url":"https://github.com/mindspore-courses/Deep-Reinforcement-Learning-Algorithms-with-MindSpore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mindspore","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"hierarchical-reinforcement-learning","task_name":"Hierarchical Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1704.03012","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}