{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/structured-control-nets-for-deep","title":"Structured Control Nets for Deep Reinforcement Learning","arxiv_id":"1802.08311","date":"2018-02-22","proceeding":"ICML 2018 7","authors":["Mario Srouji","Jian Zhang","Ruslan Salakhutdinov"],"abstract":"In recent years, Deep Reinforcement Learning has made impressive advances in\nsolving several important benchmark problems for sequential decision making.\nMany control applications use a generic multilayer perceptron (MLP) for\nnon-vision parts of the policy network. In this work, we propose a new neural\nnetwork architecture for the policy network representation that is simple yet\neffective. The proposed Structured Control Net (SCN) splits the generic MLP\ninto two separate sub-modules: a nonlinear control module and a linear control\nmodule. Intuitively, the nonlinear control is for forward-looking and global\ncontrol, while the linear control stabilizes the local dynamics around the\nresidual of global control. We hypothesize that this will bring together the\nbenefits of both linear and nonlinear policies: improve training sample\nefficiency, final episodic reward, and generalization of learned policy, while\nrequiring a smaller network and being generally applicable to different\ntraining methods. We validated our hypothesis with competitive results on\nsimulations from OpenAI MuJoCo, Roboschool, Atari, and a custom 2D urban\ndriving environment, with various ablation and generalization tests, trained\nwith multiple black-box and policy gradient training methods. The proposed\narchitecture has the potential to improve upon broader control tasks by\nincorporating problem specific priors into the architecture. As a case study,\nwe demonstrate much improved performance for locomotion tasks by emulating the\nbiological central pattern generators (CPGs) as the nonlinear part of the\narchitecture.","url_abs":"http://arxiv.org/abs/1802.08311v1","url_pdf":"http://arxiv.org/pdf/1802.08311v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"structured-control-nets-for-deep","repo_url":"https://github.com/wongongv/scnwithdqn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"mujoco","task_name":"MuJoCo"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.08311","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}