Item Response Theory for AI Safety

• URL: http://arxiv.org/abs/2608.05086v1
• Date: Wed, 05 Aug 2026 17:25:27 GMT
• Title: Item Response Theory for AI Safety
• Abstract summary: Item Response Theory (IRT) is a statistical toolkit for measuring latents from performance on items with inferred psychometric properties.<n>IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.
• Score: 23.16156306184828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.

Related papers

• The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean [2.822987192839875]
  Agent benchmarks are increasingly used to compare large language models (LLMs) and guide deployment decisions.<n>We identify a double measurement confound: execution-critical decisions are performed by a fixed scaffold instead of the model, while the scorer evaluates outputs using criteria that may not reflect task correctness.<n>We unify these issues within a measurement-theoretic framework that characterizes when benchmark scores can be interpreted as evidence of model capability.
  arXiv  Detail & Related papers  (2026-09-06T21:23:11Z)
• Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models [10.539135916505133]
  Small Language Models (SLMs) are increasingly deployed in resource-constrained, privacy-sensitive settings.<n>Existing AI safetyslash securityslash compliance benchmarks are designed for large language models that may not transfer reliably to SLMs.<n>We ask: Can these benchmarks effectively and reliably evaluate SLMs?
  arXiv  Detail & Related papers  (2026-08-17T22:47:14Z)
• Can We Trust Item Response Theory for AI Evaluation? [10.509963423246974]
  AI benchmarks increasingly leverage item-level statistical models to estimate capabilities, rank systems, select informative examples, and diagnose benchmark quality.<n>IRT tools were originally developed for human testing, but AI benchmarks often involve fewer models, far more items, and capability distributions that may be skewed, clustered, or multimodal.<n>This study examines how these regime mismatches challenge the reliability of IRT modeling for AI evaluation.
  arXiv  Detail & Related papers  (2026-07-16T16:44:40Z)
• Latent Performance Profiling of Large Language Models [47.009623327601226]
  We introduce Latent Performance Profiling (LPP) -- a framework that derives task-agnostic diagnostics from hidden activations and output distributions.<n>Unlike static accuracy scores, LPP provides stable, architecture-sensitive signatures across models of similar size.<n>We show that models with similar benchmark scores can exhibit contrasting latent profiles, such as differences in entropy or adaptability.
  arXiv  Detail & Related papers  (2026-05-28T14:41:26Z)
• Efficient Safety Benchmarking via Item Response Theory [0.4036506515685838]
  We analyze a suite of widely used safety benchmarks and make three contributions toward more efficient safety evaluation.<n>We show that Item Response Theory (IRT) recovers interpretable structure on safety benchmarks, with ability estimates resolving differences among models.<n>Second, we show that adaptive item selection, which dynamically chooses informative items for each model based on its responses, approximates full-benchmark rankings.<n>Third, we introduce a practical procedure for extracting a fixed, informative subset of items reusable across models, providing an alternative to adaptive selection.
  arXiv  Detail & Related papers  (2026-05-26T17:35:31Z)
• BenchBench: Benchmarking Automated Benchmark Generation [10.44497524694021]
  BenchBench is a pipeline and dataset for benchmarking automated benchmark generation.<n>We generate 16.7K items, retain 15K core items post-filtering, and produce 152K graded model--item responses.
  arXiv  Detail & Related papers  (2026-03-21T13:05:32Z)
• When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation [80.66788281323414]
  We analyze benchmark saturation across 60 Large Language Model (LLM) benchmarks selected from technical reports by major model developers.<n>Our analysis reveals that nearly half of the benchmarks exhibit saturation, with rates increasing as benchmarks age.<n>Expert-curated benchmarks resist saturation better than crowdsourced ones.
  arXiv  Detail & Related papers  (2026-02-18T16:51:37Z)
• Benchmark^2: Systematic Evaluation of LLM Benchmarks [66.2731798872668]
  We propose Benchmark2, a comprehensive framework comprising three complementary metrics.<n>We conduct experiments across 15 benchmarks spanning mathematics, reasoning, and knowledge domains.<n>Our analysis reveals significant quality variations among existing benchmarks and demonstrates that selective benchmark construction can achieve comparable evaluation performance.
  arXiv  Detail & Related papers  (2026-01-07T14:59:03Z)
• Fantastic Bugs and Where to Find Them in AI Benchmarks [28.604919035475188]
  We introduce a framework for systematic benchmark revision that leverages statistical analysis of response patterns to flag potentially invalid questions.<n>Our approach builds on a core assumption commonly used in AI evaluations that the mean score sufficiently summarizes model performance.<n>Across nine widely used benchmarks, our method guides expert review to identify problematic questions with up to 84% precision.
  arXiv  Detail & Related papers  (2025-11-20T22:49:21Z)
• Fluid Language Model Benchmarking [126.92394365620525]
  We introduce Fluid Benchmarking, a new evaluation approach that advances LM benchmarking across multiple dimensions.<n>Inspired by psychometrics, Fluid Benchmarking is based on the insight that the relative value of benchmark items depends on an LM's capability level.<n>We examine four dimensions -- efficiency, validity, variance, and saturation -- and find that Fluid Benchmarking achieves superior performance in all of them.
  arXiv  Detail & Related papers  (2025-09-14T05:49:42Z)
• Do Large Language Model Benchmarks Test Reliability? [66.1783478365998]
  We investigate how well current benchmarks quantify model reliability.<n>Motivated by this gap in the evaluation of reliability, we propose the concept of so-called platinum benchmarks.<n>We evaluate a wide range of models on these platinum benchmarks and find that, indeed, frontier LLMs still exhibit failures on simple tasks.
  arXiv  Detail & Related papers  (2025-02-05T18:58:19Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal [64.9938658716425]
  SORRY-Bench is a proposed benchmark for evaluating large language models' (LLMs) ability to recognize and reject unsafe user requests.<n>First, existing methods often use coarse-grained taxonomy of unsafe topics, and are over-representing some fine-grained topics.<n>Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.