{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/or-bench-an-over-refusal-benchmark-for-large","title":"OR-Bench: An Over-Refusal Benchmark for Large Language Models","arxiv_id":"2405.20947","date":"2024-05-31","proceeding":null,"authors":["Justin Cui","Wei-Lin Chiang","Ion Stoica","Cho-Jui Hsieh"],"abstract":"Large Language Models (LLMs) require careful safety alignment to prevent malicious outputs. While significant research focuses on mitigating harmful content generation, the enhanced safety often come with the side effect of over-refusal, where LLMs may reject innocuous prompts and become less helpful. Although the issue of over-refusal has been empirically observed, a systematic measurement is challenging due to the difficulty of crafting prompts that can elicit the over-refusal behaviors of LLMs. This study proposes a novel method for automatically generating large-scale over-refusal datasets. Leveraging this technique, we introduce OR-Bench, the first large-scale over-refusal benchmark. OR-Bench comprises 80,000 over-refusal prompts across 10 common rejection categories, a subset of around 1,000 hard prompts that are challenging even for state-of-the-art LLMs, and an additional 600 toxic prompts to prevent indiscriminate responses. We then conduct a comprehensive study to measure the over-refusal of 32 popular LLMs across 8 model families. Our datasets are publicly available at https://huggingface.co/bench-llms and our codebase is open-sourced at https://github.com/justincui03/or-bench. We hope this benchmark can help the community develop better safety aligned models.","url_abs":"https://arxiv.org/abs/2405.20947v3","url_pdf":"https://arxiv.org/pdf/2405.20947v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"or-bench-an-over-refusal-benchmark-for-large","repo_url":"https://github.com/justincui03/or-bench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"safety-alignment","task_name":"Safety Alignment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.20947","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.20947"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/justincui03/or-bench","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d1e3f8ced72d9ee1","entry":"check_category","repo":"justincui03/or-bench","repo_kind":"official","path":"moderator/moderate_only.py","file_url":"https://github.com/justincui03/or-bench/blob/HEAD/moderator/moderate_only.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1e3f8ced72d9ee1"}},{"code_sha256_prefix":"bba6440d88b226a5","entry":"extract_text","repo":"justincui03/or-bench","repo_kind":"official","path":"moderator/moderate_only.py","file_url":"https://github.com/justincui03/or-bench/blob/HEAD/moderator/moderate_only.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bba6440d88b226a5"}},{"code_sha256_prefix":"a3ffa48b9c836938","entry":"get_marker","repo":"justincui03/or-bench","repo_kind":"official","path":"plot.py","file_url":"https://github.com/justincui03/or-bench/blob/HEAD/plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a3ffa48b9c836938"}},{"code_sha256_prefix":"8c4e4c1d7c7dd67d","entry":"get_model_family","repo":"justincui03/or-bench","repo_kind":"official","path":"plot.py","file_url":"https://github.com/justincui03/or-bench/blob/HEAD/plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c4e4c1d7c7dd67d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}