{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safe-value-functions","title":"Safe Value Functions","arxiv_id":"2105.12204","date":"2021-05-25","proceeding":null,"authors":["Pierre-François Massiani","Steve Heim","Friedrich Solowjow","Sebastian Trimpe"],"abstract":"Safety constraints and optimality are important, but sometimes conflicting criteria for controllers. Although these criteria are often solved separately with different tools to maintain formal guarantees, it is also common practice in reinforcement learning to simply modify reward functions by penalizing failures, with the penalty treated as a mere heuristic. We rigorously examine the relationship of both safety and optimality to penalties, and formalize sufficient conditions for safe value functions (SVFs): value functions that are both optimal for a given task, and enforce safety constraints. We reveal this structure by examining when rewards preserve viability under optimal control, and show that there always exists a finite penalty that induces a safe value function. This penalty is not unique, but upper-unbounded: larger penalties do not harm optimality. Although it is often not possible to compute the minimum required penalty, we reveal clear structure of how the penalty, rewards, discount factor, and dynamics interact. This insight suggests practical, theory-guided heuristics to design reward functions for control problems where safety is important.","url_abs":"https://arxiv.org/abs/2105.12204v4","url_pdf":"https://arxiv.org/pdf/2105.12204v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safe-value-functions","repo_url":"https://github.com/sheim/vibly","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.12204","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.12204"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sheim/vibly","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":23},"by_repo_kind":{"official":{"samples":23,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"778835f0e155d64a","entry":"check_failure","repo":"sheim/vibly","repo_kind":"official","path":"models/hovership.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/hovership.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"778835f0e155d64a"}},{"code_sha256_prefix":"8477d9c9e6dac3fd","entry":"check_failure","repo":"sheim/vibly","repo_kind":"official","path":"models/satellite.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/satellite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8477d9c9e6dac3fd"}},{"code_sha256_prefix":"49d152837032f814","entry":"check_failure","repo":"sheim/vibly","repo_kind":"official","path":"models/spaceship.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/spaceship.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49d152837032f814"}},{"code_sha256_prefix":"a62ab20e7683d38d","entry":"feasible","repo":"sheim/vibly","repo_kind":"official","path":"models/daslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/daslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a62ab20e7683d38d"}},{"code_sha256_prefix":"401b526de5868422","entry":"feasible","repo":"sheim/vibly","repo_kind":"official","path":"models/nslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/nslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"401b526de5868422"}},{"code_sha256_prefix":"56e6aa0272a8e3a1","entry":"is_feasible","repo":"sheim/vibly","repo_kind":"official","path":"models/slip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/slip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56e6aa0272a8e3a1"}},{"code_sha256_prefix":"f64b3b9d09fa889a","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/hovership.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/hovership.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f64b3b9d09fa889a"}},{"code_sha256_prefix":"c24f5db901a76cd5","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/lip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/lip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c24f5db901a76cd5"}},{"code_sha256_prefix":"d5ebd8d14f7153aa","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/nslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/nslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5ebd8d14f7153aa"}},{"code_sha256_prefix":"e5a186598cf57a93","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/satellite.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/satellite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5a186598cf57a93"}},{"code_sha256_prefix":"67921ac0bbc1fa70","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/spaceship.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/spaceship.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67921ac0bbc1fa70"}},{"code_sha256_prefix":"f2800c9555bb888f","entry":"p_map","repo":"sheim/vibly","repo_kind":"official","path":"models/spaceship4.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/spaceship4.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2800c9555bb888f"}},{"code_sha256_prefix":"4ff690ea4289af59","entry":"poincare_map","repo":"sheim/vibly","repo_kind":"official","path":"models/daslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/daslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ff690ea4289af59"}},{"code_sha256_prefix":"2b1c599408cf10d9","entry":"poincare_map","repo":"sheim/vibly","repo_kind":"official","path":"models/parslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/parslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b1c599408cf10d9"}},{"code_sha256_prefix":"334f6a334d749a87","entry":"sa2xp","repo":"sheim/vibly","repo_kind":"official","path":"models/hovership.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/hovership.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"334f6a334d749a87"}},{"code_sha256_prefix":"55fec93de1d4453a","entry":"sa2xp","repo":"sheim/vibly","repo_kind":"official","path":"models/lip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/lip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55fec93de1d4453a"}},{"code_sha256_prefix":"8c393be39f60a318","entry":"sa2xp","repo":"sheim/vibly","repo_kind":"official","path":"models/satellite.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/satellite.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c393be39f60a318"}},{"code_sha256_prefix":"049a55f2c47b81f5","entry":"sa2xp","repo":"sheim/vibly","repo_kind":"official","path":"models/spaceship.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/spaceship.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"049a55f2c47b81f5"}},{"code_sha256_prefix":"3caf6e3d4e341b27","entry":"sa2xp","repo":"sheim/vibly","repo_kind":"official","path":"models/spaceship4.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/spaceship4.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3caf6e3d4e341b27"}},{"code_sha256_prefix":"b24b052bf4545bd6","entry":"sa2xp_num","repo":"sheim/vibly","repo_kind":"official","path":"models/lip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/lip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b24b052bf4545bd6"}},{"code_sha256_prefix":"8d3bf3ad5f61f6cd","entry":"step","repo":"sheim/vibly","repo_kind":"official","path":"models/daslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/daslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d3bf3ad5f61f6cd"}},{"code_sha256_prefix":"2883642f8bbd352a","entry":"step","repo":"sheim/vibly","repo_kind":"official","path":"models/nslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/nslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2883642f8bbd352a"}},{"code_sha256_prefix":"d0a938a8c361a66a","entry":"step","repo":"sheim/vibly","repo_kind":"official","path":"models/parslip.py","file_url":"https://github.com/sheim/vibly/blob/HEAD/models/parslip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0a938a8c361a66a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}