{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-reinforcement-learning-in-large-discrete","title":"Deep Reinforcement Learning in Large Discrete Action Spaces","arxiv_id":"1512.07679","date":"2015-12-24","proceeding":null,"authors":["Gabriel Dulac-Arnold","Richard Evans","Hado van Hasselt","Peter Sunehag","Timothy Lillicrap","Jonathan Hunt","Timothy Mann","Theophane Weber","Thomas Degris","Ben Coppin"],"abstract":"Being able to reason in an environment with a large number of discrete\nactions is essential to bringing reinforcement learning to a larger class of\nproblems. Recommender systems, industrial plants and language models are only\nsome of the many real-world tasks involving large numbers of discrete actions\nfor which current methods are difficult or even often impossible to apply. An\nability to generalize over the set of actions as well as sub-linear complexity\nrelative to the size of the set are both necessary to handle such tasks.\nCurrent approaches are not able to provide both of these, which motivates the\nwork in this paper. Our proposed approach leverages prior information about the\nactions to embed them in a continuous space upon which it can generalize.\nAdditionally, approximate nearest-neighbor methods allow for logarithmic-time\nlookup complexity relative to the number of actions, which is necessary for\ntime-wise tractable training. This combined approach allows reinforcement\nlearning methods to be applied to large-scale learning problems previously\nintractable with current methods. We demonstrate our algorithm's abilities on a\nseries of tasks having up to one million actions.","url_abs":"http://arxiv.org/abs/1512.07679v2","url_pdf":"http://arxiv.org/pdf/1512.07679v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-reinforcement-learning-in-large-discrete","repo_url":"https://github.com/ChangyWen/wolpertinger_ddpg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-reinforcement-learning-in-large-discrete","repo_url":"https://github.com/nikhil3456/Deep-Reinforcement-Learning-in-Large-Discrete-Action-Spaces","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1512.07679","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}