Efficient Safety Benchmarking via Item Response Theory

• URL: http://arxiv.org/abs/2606.20626v1
• Date: Tue, 26 May 2026 17:35:31 GMT
• Title: Efficient Safety Benchmarking via Item Response Theory
• Abstract summary: We analyze a suite of widely used safety benchmarks and make three contributions toward more efficient safety evaluation.<n>We show that Item Response Theory (IRT) recovers interpretable structure on safety benchmarks, with ability estimates resolving differences among models.<n>Second, we show that adaptive item selection, which dynamically chooses informative items for each model based on its responses, approximates full-benchmark rankings.<n>Third, we introduce a practical procedure for extracting a fixed, informative subset of items reusable across models, providing an alternative to adaptive selection.
• Score: 0.4036506515685838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety benchmarks for language models are typically evaluated using static paradigms that treat all items as equally informative for all models, an assumption that is particularly problematic for adversarial, highly heterogeneous safety items. Applied in full to modern benchmark suites, the current evaluation procedures would require on the order of $10^5$ responses, most of which provide little ranking signal. We analyze a suite of widely used safety benchmarks and make three contributions toward more efficient safety evaluation. First, we show that Item Response Theory (IRT) recovers interpretable structure on safety benchmarks, with ability estimates resolving differences among models that cluster at the ceiling of raw safety metrics. Second, we show that adaptive item selection, which dynamically chooses informative items for each model based on its responses, approximates full-benchmark rankings while reducing evaluation cost by at least 80% on benchmarks where Spearman's $ρ>$90% with full-benchmark is attainable, and by up to 99.9% on AIR-Bench 2024. Third, we introduce a practical procedure for extracting a fixed, informative subset of items reusable across models, providing an alternative to adaptive selection with savings of up to 99.8% on AIR-Bench 2024. Together, these results establish that psychometric methods enable benchmark-aware reductions in evaluation costs across the safety evaluation pipeline.

Related papers

• Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models [10.539135916505133]
  Small Language Models (SLMs) are increasingly deployed in resource-constrained, privacy-sensitive settings.<n>Existing AI safetyslash securityslash compliance benchmarks are designed for large language models that may not transfer reliably to SLMs.<n>We ask: Can these benchmarks effectively and reliably evaluate SLMs?
  arXiv  Detail & Related papers  (2026-08-17T22:47:14Z)
• Item Response Theory for AI Safety [23.16156306184828]
  Item Response Theory (IRT) is a statistical toolkit for measuring latents from performance on items with inferred psychometric properties.<n>IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.
  arXiv  Detail & Related papers  (2026-08-05T17:25:27Z)
• PACE: A Proxy for Agentic Capability Evaluation [60.3743414796937]
  We introduce PACE, a framework that constructs proxy benchmarks by selecting instances from existing non-agentic evaluations.<n>Experiments across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks show that PACE-Bench predicts agentic scores with leave-one-out cross-validation (LOOCV) mean absolute error (MAE) under 4%, Spearman correlation above 0.80, and pairwise model-ranking accuracy around 85%, all at much less than 1% of the full agentic evaluation cost.
  arXiv  Detail & Related papers  (2026-07-02T10:59:03Z)
• Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges [2.8523456804049885]
  Safety judges are increasingly deployed to evaluate model outputs against evolving criteria.<n>Recent meta-evaluation work shows they remain brittle under prompt and rubric variation.<n>We propose a training strategy that combines instance-conditioned dynamic rubrics and a reliable-to-expressive curriculum.
  arXiv  Detail & Related papers  (2026-06-08T08:02:57Z)
• Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents [6.787194586338237]
  We present the first systematic analysis dedicated to agent safety benchmarks as evaluation instruments.<n>A coverage matrix reveals broad risk coverage but limited methodological convergence, while the taxonomy analysis shows a behavioral-benchmark core concentrated in sandboxed, constrained, and often safety-only evaluation.<n>Across the landscape, we find that benchmark choice can yield contradictory safety conclusions, coverage counts often overstate evaluation depth, environment fidelity shapes reported safety, the field disproportionately tests externally imposed rather than agent-internal risks, metric fragmentation limits comparison, and remains effectively unbenchmarked.
  arXiv  Detail & Related papers  (2026-04-11T04:25:19Z)
• ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis [96.92417622318267]
  ATBench is a trajectory-level benchmark for structured, diverse, and realistic evaluation of agent safety.<n>It organizes agentic risk along three dimensions: risk source, failure mode, and real-world harm.<n>It contains 1,000 trajectories (503 safe and 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools.
  arXiv  Detail & Related papers  (2026-04-02T13:26:20Z)
• When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models [1.5049442691806052]
  Large Language Models (LLMs) are increasingly integrated into high-stakes applications.<n>Existing safety evaluations rely on fixed collections of harmful prompts, implicitly assuming non-adaptive adversaries.<n>In this work, we examine the vulnerability of contemporary language models to automated, adversarial prompt refinement.
  arXiv  Detail & Related papers  (2026-02-21T05:35:26Z)
• ReLE: A Scalable System and Structured Benchmark for Diagnosing Capability Anisotropy in Chinese LLMs [37.23311145049677]
  We present ReLE, a scalable system designed to diagnose Capability Anisotropy.<n>We evaluate 304 models across a Domain $times$ Capability Symbolic matrix comprising 207,843 samples.
  arXiv  Detail & Related papers  (2026-01-24T09:57:59Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• Fantastic Bugs and Where to Find Them in AI Benchmarks [28.604919035475188]
  We introduce a framework for systematic benchmark revision that leverages statistical analysis of response patterns to flag potentially invalid questions.<n>Our approach builds on a core assumption commonly used in AI evaluations that the mean score sufficiently summarizes model performance.<n>Across nine widely used benchmarks, our method guides expert review to identify problematic questions with up to 84% precision.
  arXiv  Detail & Related papers  (2025-11-20T22:49:21Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains.<n>Existing research predominantly concentrates on the security of general large language models.<n>This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal [64.9938658716425]
  SORRY-Bench is a proposed benchmark for evaluating large language models' (LLMs) ability to recognize and reject unsafe user requests.<n>First, existing methods often use coarse-grained taxonomy of unsafe topics, and are over-representing some fine-grained topics.<n>Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models [65.79770974145983]
  ASSERT, Automated Safety Scenario Red Teaming, consists of three methods -- semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. We find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings.
  arXiv  Detail & Related papers  (2023-10-14T17:10:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.