LLM Jailbreak

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

11 shown of 11 papers with code (24 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Automatic Prompt Optimization with "Gradient Descent" and Beam Search
  4 May 2023 6 repositories listed Syntology ran 5 of 9 samples · 4 unverified
  Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort.
• Derail Yourself: Multi-turn LLM Jailbreak Attack through Self-discovered Clues
  14 Oct 2024 2 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  This study exposes the safety vulnerabilities of Large Language Models (LLMs) in multi-turn interactions, where malicious users can obscure harmful intents across several queries.
• PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  20 May 2025 1 repository listed
  Large language models (LLMs) have achieved remarkable capabilities but remain vulnerable to adversarial prompts known as jailbreaks, which can bypass safety alignment and elicit harmful outputs.
• Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation
  28 Jan 2025 1 repository listed
  This paper introduces the GAP (Graph of Attacks with Pruning) framework, an advanced approach for generating stealthy jailbreak prompts to evaluate and enhance LLM safeguards.
• CySecBench: Generative AI-based CyberSecurity-focused Prompt Dataset for Benchmarking Large Language Models
  2 Jan 2025 1 repository listed
  Typically, these methods are evaluated using datasets of malicious prompts designed to bypass security policies established by LLM providers.
• SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage
  19 Dec 2024 1 repository listed
  It then employs a simple assistive task such as a masked language model task or an element lookup by position task to encode the semantics of the masked keywords.
• SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis
  21 Oct 2024 1 repository listed
  The increasing integration of large language models (LLMs) across various fields has heightened concerns about their potential to propagate dangerous information.
• Cognitive Overload Attack:Prompt Injection for Long Context
  15 Oct 2024 1 repository listed
  We applied the principles of Cognitive Load Theory in LLMs and empirically validate that similar to human cognition, LLMs also suffer from cognitive overload a state where the demand on cognitive processing exceeds the…
• JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models
  26 Jun 2024 1 repository listed
  The rapid evolution of artificial intelligence (AI) through developments in Large Language Models (LLMs) and Vision-Language Models (VLMs) has brought significant advancements across various technological domains.
• Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization
  15 May 2024 1 repository listed
  Our approach relaxes the discrete jailbreak optimization into a continuous optimization and progressively increases the sparsity of the optimizing vectors.
• JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  3 Apr 2024 1 repository listed
  With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge.

Syntology lines on 2 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.