Redteaming Resistance Benchmark

The Redteaming Resistance Benchmark is a project aimed at evaluating the robustness of language models, both open-source and black-box, through redteaming attacks. These attacks involve systematically challenging and testing models with carefully crafted prompts to uncover their failure modes and vulnerabilities. In other words, it reveals where these models are susceptible to generating problematic outputs¹².

Here are some key points about the Redteaming Resistance Benchmark:

• Purpose: The project aims to identify weaknesses in language models by simulating adversarial scenarios.
• Datasets Used: The benchmark uses several datasets to evaluate model performance, including:
  • Advbench: A dataset of adversarial behaviors ranging from profanity, discrimination, to violence.
  • AART: A collection of generated adversarial behaviors with diverse cultural, geographic, and application settings.
  • Beavertails: A dataset for safety alignment research in large language models.
  • Do Not Answer: A dataset of prompts to which responsible language models do not answer.
  • RedEval - HarmfulQA and RedEval - DangerousQA: Datasets of harmful questions covering various topics.
  • Student-Teacher Prompting: A dataset of harmful prompts that successfully broke a specific language model.
  • SAP: Attacks generated through in-context learning to mimic human speech.
• Content Categories: The benchmark evaluates model performance across 15 different categories, including illegal activity, harm, hate, discrimination, and more¹.

(1) GitHub - haizelabs/redteaming-resistance-benchmark. https://github.com/haizelabs/redteaming-resistance-benchmark. (2) Introducing the Red-Teaming Resistance Leaderboard - Hugging Face. https://huggingface.co/blog/leaderboard-haizelab. (3) Redteaming Resistance Leaderboard - a Hugging Face Space by Nymbo. https://huggingface.co/spaces/Nymbo/red-teaming-resistance-benchmark. (4) Redteaming Resistance Benchmark - GitHub. https://github.com/jussker/redteaming-resistance-benchmark/blob/main/README.md.