{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-alignment-and-robustness-with-short","title":"Improving Alignment and Robustness with Circuit Breakers","arxiv_id":"2406.04313","date":"2024-06-06","proceeding":null,"authors":["Andy Zou","Long Phan","Justin Wang","Derek Duenas","Maxwell Lin","Maksym Andriushchenko","Rowan Wang","Zico Kolter","Matt Fredrikson","Dan Hendrycks"],"abstract":"AI systems can take harmful actions and are highly vulnerable to adversarial attacks. We present an approach, inspired by recent advances in representation engineering, that interrupts the models as they respond with harmful outputs with \"circuit breakers.\" Existing techniques aimed at improving alignment, such as refusal training, are often bypassed. Techniques such as adversarial training try to plug these holes by countering specific attacks. As an alternative to refusal training and adversarial training, circuit-breaking directly controls the representations that are responsible for harmful outputs in the first place. Our technique can be applied to both text-only and multimodal language models to prevent the generation of harmful outputs without sacrificing utility -- even in the presence of powerful unseen attacks. Notably, while adversarial robustness in standalone image recognition remains an open challenge, circuit breakers allow the larger multimodal system to reliably withstand image \"hijacks\" that aim to produce harmful content. Finally, we extend our approach to AI agents, demonstrating considerable reductions in the rate of harmful actions when they are under attack. Our approach represents a significant step forward in the development of reliable safeguards to harmful behavior and adversarial attacks.","url_abs":"https://arxiv.org/abs/2406.04313v4","url_pdf":"https://arxiv.org/pdf/2406.04313v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-alignment-and-robustness-with-short","repo_url":"https://github.com/blackswan-ai/circuit-breakers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"improving-alignment-and-robustness-with-short","repo_url":"https://github.com/blackswan-ai/short-circuiting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"improving-alignment-and-robustness-with-short","repo_url":"https://github.com/grayswanai/circuit-breakers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"improving-alignment-and-robustness-with-short","repo_url":"https://github.com/schwinnl/circuit-breakers-eval","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.04313","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.04313"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/grayswanai/circuit-breakers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/blackswan-ai/circuit-breakers","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/schwinnl/circuit-breakers-eval","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/blackswan-ai/short-circuiting","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":2},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b5b12432e2c1124d","entry":"compute_loss","repo":"grayswanai/circuit-breakers","repo_kind":"official","path":"src/lorra_circuit_breaker.py","file_url":"https://github.com/grayswanai/circuit-breakers/blob/HEAD/src/lorra_circuit_breaker.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5b12432e2c1124d"}},{"code_sha256_prefix":"257ec7d5d8d6c5b6","entry":"handle_non_serializable","repo":"blackswan-ai/circuit-breakers","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/blackswan-ai/circuit-breakers/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"257ec7d5d8d6c5b6"}},{"code_sha256_prefix":"932da9d6ec274164","entry":"mellowmax","repo":"schwinnl/circuit-breakers-eval","repo_kind":"listed","path":"evaluation/softopt.py","file_url":"https://github.com/schwinnl/circuit-breakers-eval/blob/HEAD/evaluation/softopt.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"932da9d6ec274164"}},{"code_sha256_prefix":"ae03a7f84547715f","entry":"init_rep_control","repo":"blackswan-ai/circuit-breakers","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/blackswan-ai/circuit-breakers/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae03a7f84547715f"}},{"code_sha256_prefix":"2671d81aefc6945e","entry":"load_model_and_tokenizer","repo":"blackswan-ai/circuit-breakers","repo_kind":"official","path":"evaluation/utils.py","file_url":"https://github.com/blackswan-ai/circuit-breakers/blob/HEAD/evaluation/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2671d81aefc6945e"}},{"code_sha256_prefix":"a28e636efca9a712","entry":"run","repo":"blackswan-ai/circuit-breakers","repo_kind":"official","path":"evaluation/softopt.py","file_url":"https://github.com/blackswan-ai/circuit-breakers/blob/HEAD/evaluation/softopt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a28e636efca9a712"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}