{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-reinforcement-learning-with-feedback","title":"Deep Reinforcement Learning with Feedback-based Exploration","arxiv_id":"1903.06151","date":"2019-03-14","proceeding":null,"authors":["Jan Scholten","Daan Wout","Carlos Celemin","Jens Kober"],"abstract":"Deep Reinforcement Learning has enabled the control of increasingly complex\nand high-dimensional problems. However, the need of vast amounts of data before\nreasonable performance is attained prevents its widespread application. We\nemploy binary corrective feedback as a general and intuitive manner to\nincorporate human intuition and domain knowledge in model-free machine\nlearning. The uncertainty in the policy and the corrective feedback is combined\ndirectly in the action space as probabilistic conditional exploration. As a\nresult, the greatest part of the otherwise ignorant learning process can be\navoided. We demonstrate the proposed method, Predictive Probabilistic Merging\nof Policies (PPMP), in combination with DDPG. In experiments on continuous\ncontrol problems of the OpenAI Gym, we achieve drastic improvements in sample\nefficiency, final performance, and robustness to erroneous feedback, both for\nhuman and synthetic feedback. Additionally, we show solutions beyond the\ndemonstrated knowledge.","url_abs":"http://arxiv.org/abs/1903.06151v1","url_pdf":"http://arxiv.org/pdf/1903.06151v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-reinforcement-learning-with-feedback","repo_url":"https://github.com/janscholten/ppmp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"deep-reinforcement-learning-with-feedback","repo_url":"https://github.com/pemami4911/deep-rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"continuous-control","task_name":"continuous-control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}