{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safe-exploration-in-continuous-action-spaces","title":"Safe Exploration in Continuous Action Spaces","arxiv_id":"1801.08757","date":"2018-01-26","proceeding":null,"authors":["Gal Dalal","Krishnamurthy Dvijotham","Matej Vecerik","Todd Hester","Cosmin Paduraru","Yuval Tassa"],"abstract":"We address the problem of deploying a reinforcement learning (RL) agent on a\nphysical system such as a datacenter cooling unit or robot, where critical\nconstraints must never be violated. We show how to exploit the typically smooth\ndynamics of these systems and enable RL algorithms to never violate constraints\nduring learning. Our technique is to directly add to the policy a safety layer\nthat analytically solves an action correction formulation per each state. The\nnovelty of obtaining an elegant closed-form solution is attained due to a\nlinearized model, learned on past trajectories consisting of arbitrary actions.\nThis is to mimic the real-world circumstances where data logs were generated\nwith a behavior policy that is implausible to describe mathematically; such\ncases render the known safety-aware off-policy methods inapplicable. We\ndemonstrate the efficacy of our approach on new representative physics-based\nenvironments, and prevail where reward shaping fails by maintaining zero\nconstraint violations.","url_abs":"http://arxiv.org/abs/1801.08757v1","url_pdf":"http://arxiv.org/pdf/1801.08757v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/AgrawalAmey/safe-explorer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/huiyulhy/safe-control-gym","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/kollerlukas/safe-explorer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/ustc-arg/safe-robot-learning-competition","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/utiasDSL/safe-control-gym","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"safe-exploration-in-continuous-action-spaces","repo_url":"https://github.com/zlr20/saferl_kit","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"safe-exploration","task_name":"Safe Exploration"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.08757","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1801.08757"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kollerlukas/safe-explorer","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AgrawalAmey/safe-explorer","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/utiasDSL/safe-control-gym","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huiyulhy/safe-control-gym","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zlr20/saferl_kit","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ustc-arg/safe-robot-learning-competition","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c1793cbf39af800c","entry":"eval_policy","repo":"zlr20/saferl_kit","repo_kind":"listed","path":"saferl_algos/safetylayer.py","file_url":"https://github.com/zlr20/saferl_kit/blob/HEAD/saferl_algos/safetylayer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1793cbf39af800c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}