{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reward-constrained-policy-optimization","title":"Reward Constrained Policy Optimization","arxiv_id":"1805.11074","date":"2018-05-28","proceeding":"ICLR 2019 5","authors":["Chen Tessler","Daniel J. Mankowitz","Shie Mannor"],"abstract":"Solving tasks in Reinforcement Learning is no easy feat. As the goal of the\nagent is to maximize the accumulated reward, it often learns to exploit\nloopholes and misspecifications in the reward signal resulting in unwanted\nbehavior. While constraints may solve this issue, there is no closed form\nsolution for general constraints. In this work we present a novel\nmulti-timescale approach for constrained policy optimization, called `Reward\nConstrained Policy Optimization' (RCPO), which uses an alternative penalty\nsignal to guide the policy towards a constraint satisfying one. We prove the\nconvergence of our approach and provide empirical evidence of its ability to\ntrain constraint satisfying policies.","url_abs":"http://arxiv.org/abs/1805.11074v3","url_pdf":"http://arxiv.org/pdf/1805.11074v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reward-constrained-policy-optimization","repo_url":"https://github.com/HaozheJasper/CBRL_KDD22","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"safe-reinforcement-learning","task_name":"Safe Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.11074","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}