{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-safety-constraints-for-large","title":"Learning Safety Constraints for Large Language Models","arxiv_id":"2505.24445","date":"2025-05-30","proceeding":null,"authors":["Xin Chen","Yarden As","Andreas Krause"],"abstract":"Large language models (LLMs) have emerged as powerful tools but pose significant safety risks through harmful outputs and vulnerability to adversarial attacks. We propose SaP, short for Safety Polytope, a geometric approach to LLM safety that learns and enforces multiple safety constraints directly in the model's representation space. We develop a framework that identifies safe and unsafe regions via the polytope's facets, enabling both detection and correction of unsafe outputs through geometric steering. Unlike existing approaches that modify model weights, SaP operates post-hoc in the representation space, preserving model capabilities while enforcing safety constraints. Experiments across multiple LLMs demonstrate that our method can effectively detect unethical inputs, reduce adversarial attack success rates while maintaining performance on standard tasks, thus highlighting the importance of having an explicit geometric model for safety. Analysis of the learned polytope facets reveals emergence of specialization in detecting different semantic notions of safety, providing interpretable insights into how safety is captured in LLMs' representation space.","url_abs":"https://arxiv.org/abs/2505.24445v1","url_pdf":"https://arxiv.org/pdf/2505.24445v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-safety-constraints-for-large","repo_url":"https://github.com/lasgroup/safetypolytope","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-attack","task_name":"Adversarial Attack"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.24445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.24445"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lasgroup/safetypolytope","reach":null}],"summary":{"ran_violates":1,"ran":3,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"2a97f14c0d2923cf","entry":"escape_category","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/run_beaver_pipeline.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/run_beaver_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a97f14c0d2923cf"}},{"code_sha256_prefix":"f64db1e809bc29e1","entry":"PolytopeConstraint","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/lm_constraints.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/lm_constraints.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f64db1e809bc29e1"}},{"code_sha256_prefix":"cb9031bc64ebd191","entry":"get_model_hidden_states","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/lm_constraints.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/lm_constraints.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb9031bc64ebd191"}},{"code_sha256_prefix":"4bdff8ec143ae9de","entry":"get_model_hidden_states_loop","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/lm_constraints.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/lm_constraints.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4bdff8ec143ae9de"}},{"code_sha256_prefix":"62806d4fb43f035c","entry":"get_model_name","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/run_beaver_pipeline.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/run_beaver_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62806d4fb43f035c"}},{"code_sha256_prefix":"625159cc26264cd8","entry":"ConstraintOutputs","repo":"lasgroup/safetypolytope","repo_kind":"official","path":"src/safety_polytope/polytope/lm_constraints.py","file_url":"https://github.com/lasgroup/safetypolytope/blob/HEAD/src/safety_polytope/polytope/lm_constraints.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"625159cc26264cd8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}