{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/oasis-conditional-distribution-shaping-for","title":"OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learning","arxiv_id":"2407.14653","date":"2024-07-19","proceeding":null,"authors":["Yihang Yao","Zhepeng Cen","Wenhao Ding","Haohong Lin","Shiqi Liu","Tingnan Zhang","Wenhao Yu","Ding Zhao"],"abstract":"Offline safe reinforcement learning (RL) aims to train a policy that satisfies constraints using a pre-collected dataset. Most current methods struggle with the mismatch between imperfect demonstrations and the desired safe and rewarding performance. In this paper, we introduce OASIS (cOnditionAl diStributIon Shaping), a new paradigm in offline safe RL designed to overcome these critical limitations. OASIS utilizes a conditional diffusion model to synthesize offline datasets, thus shaping the data distribution toward a beneficial target domain. Our approach makes compliance with safety constraints through effective data utilization and regularization techniques to benefit offline safe RL training. Comprehensive evaluations on public benchmarks and varying datasets showcase OASIS's superiority in benefiting offline safe RL agents to achieve high-reward behavior while satisfying the safety constraints, outperforming established baselines. Furthermore, OASIS exhibits high data efficiency and robustness, making it suitable for real-world applications, particularly in tasks where safety is imperative and high-quality demonstrations are scarce.","url_abs":"https://arxiv.org/abs/2407.14653v1","url_pdf":"https://arxiv.org/pdf/2407.14653v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"oasis-conditional-distribution-shaping-for","repo_url":"https://github.com/yihangyao/OASIS","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"safe-reinforcement-learning","task_name":"Safe Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"oasis","method_name":"OASIS"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.14653","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.14653"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yihangyao/OASIS","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"572c07ae4870962e","entry":"CUDA","repo":"yihangyao/OASIS","repo_kind":"official","path":"Generation/causal_model.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/Generation/causal_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"572c07ae4870962e"}},{"code_sha256_prefix":"cb1929149ecc4471","entry":"mlp","repo":"yihangyao/OASIS","repo_kind":"official","path":"OSRL/osrl/common/net.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/OSRL/osrl/common/net.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cb1929149ecc4471"}},{"code_sha256_prefix":"40840130a00e53c9","entry":"temp_sigmoid","repo":"yihangyao/OASIS","repo_kind":"official","path":"Generation/causal_model.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/Generation/causal_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"40840130a00e53c9"}},{"code_sha256_prefix":"134173215340e0d1","entry":"to_torch","repo":"yihangyao/OASIS","repo_kind":"official","path":"OSRL/osrl/common/net.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/OSRL/osrl/common/net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"134173215340e0d1"}},{"code_sha256_prefix":"6f1537c5f45a3b7e","entry":"download_dataset_from_url","repo":"yihangyao/OASIS","repo_kind":"official","path":"DSRL/dsrl/offline_env.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/DSRL/dsrl/offline_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6f1537c5f45a3b7e"}},{"code_sha256_prefix":"09c8479d9a5b3e06","entry":"extract","repo":"yihangyao/OASIS","repo_kind":"official","path":"OSRL/osrl/common/net.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/OSRL/osrl/common/net.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"09c8479d9a5b3e06"}},{"code_sha256_prefix":"e14321aa0b5c0b78","entry":"filepath_from_url","repo":"yihangyao/OASIS","repo_kind":"official","path":"DSRL/dsrl/offline_env.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/DSRL/dsrl/offline_env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e14321aa0b5c0b78"}},{"code_sha256_prefix":"395635db622171fa","entry":"get_keys","repo":"yihangyao/OASIS","repo_kind":"official","path":"DSRL/dsrl/offline_env.py","file_url":"https://github.com/yihangyao/OASIS/blob/HEAD/DSRL/dsrl/offline_env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"395635db622171fa"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}