Benchmark
HarmBench
A standardized evaluation framework for red teaming language models, measuring attack success rates across direct requests and automated jailbreak methods. Lower attack success rates indicate stronger resistance to adversarial prompts.
What it measures
Attack success rate (ASR %) across multiple attack vectors: direct requests, GCG, AutoDAN, PAIR, TAP, and multimodal attacks across 510 harmful behaviors
Safety areas: Harmful content and Robustness
Score direction: lower_is_better
Public model scores
No comparable public model scores found yet.
Primary source
HarmBench: A Standardized Evaluation Framework for Automated Red TeamingStatus: Verified. Last verified: 2026-06-13.