{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/more-for-less-safe-policy-improvement-with","title":"More for Less: Safe Policy Improvement With Stronger Performance Guarantees","arxiv_id":"2305.07958","date":"2023-05-13","proceeding":null,"authors":["Patrick Wienhöft","Marnix Suilen","Thiago D. Simão","Clemens Dubslaff","Christel Baier","Nils Jansen"],"abstract":"In an offline reinforcement learning setting, the safe policy improvement (SPI) problem aims to improve the performance of a behavior policy according to which sample data has been generated. State-of-the-art approaches to SPI require a high number of samples to provide practical probabilistic guarantees on the improved policy's performance. We present a novel approach to the SPI problem that provides the means to require less data for such guarantees. Specifically, to prove the correctness of these guarantees, we devise implicit transformations on the data set and the underlying environment model that serve as theoretical foundations to derive tighter improvement bounds for SPI. Our empirical evaluation, using the well-established SPI with baseline bootstrapping (SPIBB) algorithm, on standard benchmarks shows that our method indeed significantly reduces the sample complexity of the SPIBB algorithm.","url_abs":"https://arxiv.org/abs/2305.07958v1","url_pdf":"https://arxiv.org/pdf/2305.07958v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"more-for-less-safe-policy-improvement-with","repo_url":"https://github.com/lava-lab/improved_spi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2305.07958","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.07958"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lava-lab/improved_spi","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/LAVA-LAB/improved","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7c7e0825f52f3a41","entry":"compute_Nwedge_2s","repo":"lava-lab/improved_spi","repo_kind":"official","path":"guarantee_functions.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/guarantee_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c7e0825f52f3a41"}},{"code_sha256_prefix":"42308e19a6da852c","entry":"compute_Nwedge_beta","repo":"lava-lab/improved_spi","repo_kind":"official","path":"guarantee_functions.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/guarantee_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42308e19a6da852c"}},{"code_sha256_prefix":"4e81fab908167675","entry":"compute_zeta_spibb","repo":"lava-lab/improved_spi","repo_kind":"official","path":"guarantee_functions.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/guarantee_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e81fab908167675"}},{"code_sha256_prefix":"c5a02592fcea1dd2","entry":"fast_step","repo":"lava-lab/improved_spi","repo_kind":"official","path":"env_util.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/env_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5a02592fcea1dd2"}},{"code_sha256_prefix":"508a316ff79e57cd","entry":"generate_batch","repo":"lava-lab/improved_spi","repo_kind":"official","path":"env_util.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/env_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"508a316ff79e57cd"}},{"code_sha256_prefix":"689e83fa371be0c2","entry":"policy_evaluation_modified","repo":"lava-lab/improved_spi","repo_kind":"official","path":"HCPI.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/HCPI.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"689e83fa371be0c2"}},{"code_sha256_prefix":"00a69808cc4ed02d","entry":"step","repo":"lava-lab/improved_spi","repo_kind":"official","path":"env_util.py","file_url":"https://github.com/lava-lab/improved_spi/blob/HEAD/env_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00a69808cc4ed02d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}