{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/crop-certifying-robust-policies-for","title":"CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing","arxiv_id":"2106.09292","date":"2021-06-17","proceeding":"ICLR 2022 4","authors":["Fan Wu","Linyi Li","Zijian Huang","Yevgeniy Vorobeychik","Ding Zhao","Bo Li"],"abstract":"As reinforcement learning (RL) has achieved great success and been even adopted in safety-critical domains such as autonomous vehicles, a range of empirical studies have been conducted to improve its robustness against adversarial attacks. However, how to certify its robustness with theoretical guarantees still remains challenging. In this paper, we present the first unified framework CROP (Certifying Robust Policies for RL) to provide robustness certification on both action and reward levels. In particular, we propose two robustness certification criteria: robustness of per-state actions and lower bound of cumulative rewards. We then develop a local smoothing algorithm for policies derived from Q-functions to guarantee the robustness of actions taken along the trajectory; we also develop a global smoothing algorithm for certifying the lower bound of a finite-horizon cumulative reward, as well as a novel local smoothing algorithm to perform adaptive search in order to obtain tighter reward certification. Empirically, we apply CROP to evaluate several existing empirically robust RL algorithms, including adversarial training and different robust regularization, in four environments (two representative Atari games, Highway, and CartPole). Furthermore, by evaluating these algorithms against adversarial attacks, we demonstrate that our certification are often tight. All experiment results are available at website https://crop-leaderboard.github.io.","url_abs":"https://arxiv.org/abs/2106.09292v2","url_pdf":"https://arxiv.org/pdf/2106.09292v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"crop-certifying-robust-policies-for","repo_url":"https://github.com/ai-secure/crop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"crop-certifying-robust-policies-for","repo_url":"https://github.com/zijianh4/CROP-leaderboard.github.io","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2106.09292","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.09292"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/AI-secure/CROP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zijianh4/CROP-leaderboard.github.io","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ai-secure/crop","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"ec247a75cc72177f","entry":"Elem","repo":"ai-secure/crop","repo_kind":"official","path":"lo_re.py","file_url":"https://github.com/ai-secure/crop/blob/HEAD/lo_re.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec247a75cc72177f"}},{"code_sha256_prefix":"7cfbf471bbfde1c6","entry":"LoAct","repo":"ai-secure/crop","repo_kind":"official","path":"lo_re.py","file_url":"https://github.com/ai-secure/crop/blob/HEAD/lo_re.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7cfbf471bbfde1c6"}},{"code_sha256_prefix":"0e1993984d05699b","entry":"LoRe","repo":"ai-secure/crop","repo_kind":"official","path":"lo_re.py","file_url":"https://github.com/ai-secure/crop/blob/HEAD/lo_re.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0e1993984d05699b"}},{"code_sha256_prefix":"9f60bc72c2392b92","entry":"PrioritizedItem","repo":"ai-secure/crop","repo_kind":"official","path":"lo_re.py","file_url":"https://github.com/ai-secure/crop/blob/HEAD/lo_re.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f60bc72c2392b92"}},{"code_sha256_prefix":"ab585ce720204361","entry":"to_str","repo":"ai-secure/crop","repo_kind":"official","path":"lo_re.py","file_url":"https://github.com/ai-secure/crop/blob/HEAD/lo_re.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab585ce720204361"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}