{"url":"/dataset/safetybench","name":"SafetyBench","full_name":null,"description_markdown":"**SafetyBench** is a comprehensive benchmark designed to evaluate the safety of large language models (LLMs) using multiple-choice questions. As LLMs become increasingly prevalent, concerns about their safety have grown. SafetyBench addresses this by providing a reliable evaluation framework for researchers and developers. Here are the key points about SafetyBench:\r\n\r\n1. **Purpose**: SafetyBench aims to help researchers and developers better understand and assess the safety of LLMs. It serves as a reference for model selection and optimization, promoting the development of safe, responsible, and ethical large models that align with legislative norms, social standards, and human values¹².\r\n\r\n2. **Comprehensive Benchmark**: SafetyBench comprises **11,435 diverse multiple-choice questions** across **7 distinct categories** related to safety concerns. These questions cover a wide range of topics, allowing for thorough evaluation of LLM safety¹.\r\n\r\n3. **Multilingual Evaluation**: SafetyBench includes both **Chinese and English data**, making it suitable for evaluating LLMs in both languages. Researchers can assess model safety across different linguistic contexts¹.\r\n\r\n4. **Performance Insights**: Extensive tests using SafetyBench on **25 popular Chinese and English LLMs** (including zero-shot and few-shot settings) revealed that **GPT-4** outperformed its counterparts. However, there is still room for improvement in enhancing the safety of existing LLMs¹.\r\n\r\n5. **Availability**: Data and evaluation guidelines for SafetyBench are accessible through the following URLs:\r\n   - [SafetyBench Data and Guidelines](https://arxiv.org/abs/2309.07045)\r\n   - [SafetyBench Submission Entrance and Leaderboard](https://arxiv.org/abs/2309.07045)¹⁴.\r\n\r\nIn summary, SafetyBench provides a valuable resource for assessing and advancing the safety of large language models, contributing to their responsible deployment and alignment with societal norms and values.\r\n\r\n(1) [2309.07045] SafetyBench: Evaluating the Safety of Large Language .... https://arxiv.org/abs/2309.07045.\r\n(2) SafetyBench：通过单选题评估大型语言模型安全性. https://posts.careerengine.us/p/6511afb61a8da974e9d62f40.\r\n(3) thu-coai/SafetyBench · Datasets at Hugging Face. https://huggingface.co/datasets/thu-coai/SafetyBench.\r\n(4) SafetyBench：通过单选题评估大型语言模型安全性_鲟曦研习社. https://www.kuxai.com/article/1505.\r\n(5) GitHub - thu-coai/SafetyBench: Official github repo for SafetyBench, a .... https://github.com/thu-coai/SafetyBench.\r\n(6) undefined. https://doi.org/10.48550/arXiv.2309.07045.","description_withheld":null,"homepage":"https://github.com/thu-coai/SafetyBench","introduced_date":"2023-09-13","introduced_date_note":null,"introduced_by":{"paper":"/paper/safetybench-evaluating-the-safety-of-large","title":"SafetyBench: Evaluating the Safety of Large Language Models","first_author":"Zhexin Zhang","url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["SafetyBench"],"data_loaders":[],"num_papers_in_archive":28,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-25T09:33:49+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}