AIR-BENCH Live: An Evolving Safety Benchmark for Foundation Models

• URL: http://arxiv.org/abs/2607.22671v1
• Date: Mon, 06 Jul 2026 17:26:46 GMT
• Title: AIR-BENCH Live: An Evolving Safety Benchmark for Foundation Models
• Abstract summary: We present AIR-BENCH Live, a self-evolving successor to AIR-BENCH 2024.<n>An automated update pipeline monitors government regulation and classifies new policies against the current four-tier risk taxonomy.<n>A multi-agent, persona-driven prompt generation algorithm generates realistic, multilingual prompts with minimal human review.
• Score: 8.293531821026955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation-model safety benchmarks capture the AI risks of their time of publication: as models improve and governments pass new AI-safety legislation, their risk taxonomies become incomprehensive and their attack prompts become ineffective. We present AIR-BENCH Live, a self-evolving successor to AIR-BENCH 2024. An automated update pipeline monitors government regulation and classifies new policies against the current four-tier risk taxonomy, either matching them to existing categories or proposing new granular categories. Then, a multi-agent, persona-driven prompt generation algorithm generates realistic, multilingual prompts with minimal human review, leaving room for improvement with modern jail breaking techniques. This algorithm is used to overhaul legacy prompts and generate prompts for new categories. In our current version, the pipeline has expanded the benchmark from 314 to 335 granular risks, with the 21 new categories drawing from 31 truly novel policy clauses across seven jurisdictions. Evaluating 14 recent models, we find a wide safety spread (from 0.17 to 0.89 among the models judged on their own behavior), that the modernized prompts are on average 0.06 points harder than the 2024 set, with the largest drops concentrated among the most compliant models, and that most models are modestly less safe on non-English prompts. By continuously absorbing new regulation and regenerating prompts, AIR-BENCH Live is designed to evolve alongside a fast-moving field.

Related papers

• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety [2.661610409070365]
  We introduce Boiling the Frog, a benchmark that evaluates whether tool-using AI models are susceptible to incremental attacks.<n> Scenarios are organized through a three-level operational risk taxonomy.<n>Across a nine-model panel, aggregate strict attack success rate (ASR) is 44.4%.
  arXiv  Detail & Related papers  (2026-05-21T15:50:18Z)
• Selective Safety Steering via Value-Filtered Decoding [54.87935112120107]
  Large language models (LLMs) are trained to align with human values, but their generations may still violate safety constraints.<n>Existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model.<n>We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of unsafe responses.
  arXiv  Detail & Related papers  (2026-05-14T12:13:08Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• When AI Fails, What Works? A Data-Driven Taxonomy of Real-World AI Risk Mitigation Strategies [0.04736448323490553]
  We analyze real-world AI incident reporting and mitigation actions to derive an empirically grounded taxonomy.<n>Using a unified corpus of 9,705 media-reported AI incident articles, we extract explicit mitigation actions from 6,893 texts.<n>Our taxonomy introduces four new mitigation categories, including 1) Corrective and Restrictive Actions, 2) Legal/Regulatory Enforcement Actions, 3) Financial, Economic, and Market Controls, and 4) Avoidance and Denial.
  arXiv  Detail & Related papers  (2026-03-04T16:46:13Z)
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition [101.86739402748995]
  We run the largest public red-teaming competition to date, targeting 22 frontier AI agents across 44 realistic deployment scenarios.<n>We build the Agent Red Teaming benchmark and evaluate it across 19 state-of-the-art models.<n>Our findings highlight critical and persistent vulnerabilities in today's AI agents.
  arXiv  Detail & Related papers  (2025-07-28T05:13:04Z)
• Safety Pretraining: Toward the Next Generation of Safe AI [68.99129474671282]
  We present a data-centric pretraining framework that builds safety into the model from the start.<n>Our framework consists of four key steps: Safety Filtering, Safety Rephrasing, Native Refusal and Harmfulness-Tag annotated pretraining.<n>Our safety-pretrained models reduce attack success rates from 38.8% to 8.4% on standard LLM safety benchmarks with no performance on general degradation tasks.
  arXiv  Detail & Related papers  (2025-04-23T17:58:08Z)
• Auction-Based Regulation for Artificial Intelligence [28.86995747151915]
  Regulators have moved slowly to pick up the safety, bias, and legal debris left in the wake of broken AI deployment.<n>We propose an auction-based regulatory mechanism that provably incentivizes devices to deploy compliant models.<n>We show that our regulatory auction boosts compliance rates by 20% and participation rates by 15% compared to baseline regulatory mechanisms.
  arXiv  Detail & Related papers  (2024-10-02T17:57:02Z)
• AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies [80.90138009539004]
  AIR-Bench 2024 is the first AI safety benchmark aligned with emerging government regulations and company policies. It decomposes 8 government regulations and 16 company policies into a four-tiered safety taxonomy with granular risk categories in the lowest tier. We evaluate leading language models on AIR-Bench 2024, uncovering insights into their alignment with specified safety concerns.
  arXiv  Detail & Related papers  (2024-07-11T21:16:48Z)
• AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies [88.32153122712478]
  We identify 314 unique risk categories organized into a four-tiered taxonomy. At the highest level, this taxonomy encompasses System & Operational Risks, Content Safety Risks, Societal Risks, and Legal & Rights Risks. We aim to advance AI safety through information sharing across sectors and the promotion of best practices in risk mitigation for generative AI models and systems.
  arXiv  Detail & Related papers  (2024-06-25T18:13:05Z)
• Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models [11.05542708910122]
  We present a novel framework called HATEGUARD for effectively moderating new waves of online hate. Our work highlights the severe threat posed by the emergence of new waves of online hate and represents a paradigm shift in addressing this threat practically.
  arXiv  Detail & Related papers  (2023-12-22T22:34:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.