ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis

• URL: http://arxiv.org/abs/2604.02022v2
• Date: Wed, 08 Apr 2026 07:11:07 GMT
• Title: ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
• Abstract summary: ATBench is a trajectory-level benchmark for structured, diverse, and realistic evaluation of agent safety.<n>It organizes agentic risk along three dimensions: risk source, failure mode, and real-world harm.<n>It contains 1,000 trajectories (503 safe and 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools.
• Score: 96.92417622318267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or final responses. Existing trajectory-level benchmarks remain limited by insufficient interaction diversity, coarse observability of safety failures, and weak long-horizon realism. We introduce ATBench, a trajectory-level benchmark for structured, diverse, and realistic evaluation of agent safety. ATBench organizes agentic risk along three dimensions: risk source, failure mode, and real-world harm. Based on this taxonomy, we construct trajectories with heterogeneous tool pools and a long-context delayed-trigger protocol that captures realistic risk emergence across multiple stages. The benchmark contains 1,000 trajectories (503 safe and 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools. Data quality is supported by rule-based and LLM-based filtering plus full human audit. Experiments on frontier LLMs, open-source models, and specialized guard systems show that ATBench is challenging even for strong evaluators, while enabling taxonomy-stratified analysis, cross-benchmark comparison, and diagnosis of long-horizon failure patterns.

Related papers

• BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents [20.217145643468598]
  We introduce Blindspot, a benchmark for trajectory-level safety calibration of long-horizon tool-using agents.<n>Blindspot evaluates complete user-agent-environment trajectories through adaptive adversarial interaction, stateful tool execution, and execution-grounded adjudication.<n>We evaluate 13 proprietary and open-weight LLMs using eight metrics covering unsafe completion, appropriate refusal, benign utility, over-refusal, repeated-run robustness, and post-refusal failure.
  arXiv  Detail & Related papers  (2026-09-14T20:12:27Z)
• SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models [73.04773649437375]
  Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior.<n>We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale.<n>We also construct SafeAtlas-Bench, a set of 5,000 instances for evaluating five-level predictions and continuous risk scores.
  arXiv  Detail & Related papers  (2026-08-29T07:08:01Z)
• GuardianAgentBench: Where Agents Fail and How to Guard Them [1.5223648595073074]
  We present GuardianAgentBench, a benchmark of 580 scenarios across six domains evaluated on three production-ready frameworks.<n>We show that even the strongest configuration achieves only 74.8% overall accuracy and expose two distinct failure regimes.<n>Our guardrail implementation consistently outperforms system-prompt-based defenses across all models, recovering 19.9% of failures at a false positive rate of just 0.5%.
  arXiv  Detail & Related papers  (2026-07-23T07:05:05Z)
• Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents [6.787194586338237]
  We present the first systematic analysis dedicated to agent safety benchmarks as evaluation instruments.<n>A coverage matrix reveals broad risk coverage but limited methodological convergence, while the taxonomy analysis shows a behavioral-benchmark core concentrated in sandboxed, constrained, and often safety-only evaluation.<n>Across the landscape, we find that benchmark choice can yield contradictory safety conclusions, coverage counts often overstate evaluation depth, environment fidelity shapes reported safety, the field disproportionately tests externally imposed rather than agent-internal risks, metric fragmentation limits comparison, and remains effectively unbenchmarked.
  arXiv  Detail & Related papers  (2026-04-11T04:25:19Z)
• TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories [20.868825285848196]
  Safety guardrails are well-benchmarked for natural language responses, but their efficacy remains largely unexplored within multi-step tool-use trajectories.<n>To address this gap, we introduce StructuralSafe-Bench, the first comprehensive benchmark specifically designed to assess mid-trajectory safety.
  arXiv  Detail & Related papers  (2026-04-08T15:46:14Z)
• AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation [22.833977971686966]
  We present AutoControl Arena, an automated framework for frontier AI risk evaluation.<n>We mitigate hallucination while maintaining flexibility.<n>This principle, instantiated through a three-agent framework, achieves over 98% end-to-end success and 60% human preference over existing simulators.
  arXiv  Detail & Related papers  (2026-03-08T02:49:45Z)
• GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory [8.38197391178077]
  We introduce GT-HarmBench, a benchmark of 2,009 high-stakes scenarios spanning game-theoretic structures.<n>We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures.<n>Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments.
  arXiv  Detail & Related papers  (2026-02-12T17:29:52Z)
• DRAFT: Task Decoupled Latent Reasoning for Agent Safety [59.46137757545185]
  We propose DRAFT (Task Decoupled Latent Reasoning for Agent Safety), a latent reasoning framework that decouples safety judgment into two trainable stages.<n>An Extractor distills the full trajectory into a compact continuous latent draft, and a Reasoner that jointly attends to the draft and the original trajectory to predict safety.<n>DRAFT consistently outperforms strong baselines, improving accuracy from 63.27% (LoRA) to 91.18% averaged over benchmarks.
  arXiv  Detail & Related papers  (2026-02-11T07:45:14Z)
• RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories [58.32028251925354]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area.<n>We introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories.
  arXiv  Detail & Related papers  (2026-01-30T08:29:01Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models [107.82336341926134]
  SALAD-Bench is a safety benchmark specifically designed for evaluating Large Language Models (LLMs) It transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.
  arXiv  Detail & Related papers  (2024-02-07T17:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.