Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework

• URL: http://arxiv.org/abs/2605.01604v1
• Date: Sat, 02 May 2026 21:02:08 GMT
• Title: Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework
• Abstract summary: We present a taxonomy of seven failure modes unique to production agentic systems.<n>Standard metrics fail to detect four of the seven failure modes entirely.<n>We propose PAEF, a five-dimension evaluation framework with an open-source reference implementation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing evaluation frameworks for large language models -- including HELM, MT-Bench, AgentBench, and BIG-bench -- are designed for controlled, single-session, lab-scale settings. They do not address the evaluation challenges that emerge when agentic AI systems operate continuously in production: compounding decision errors, tool failure cascades, non-deterministic output drift, and the absence of ground truth for long-horizon tasks. This paper makes three contributions. First, we present a taxonomy of seven failure modes unique to production agentic systems, each grounded in observations from systems operating at billion-event scale. Second, we demonstrate empirically where standard metrics -- ROUGE, BERTScore, accuracy/AUC, and the agentic benchmarks above -- fail to detect each failure mode. Third, we propose PAEF (Production Agentic Evaluation Framework), a five-dimension evaluation framework with an open-source reference implementation, designed for continuous evaluation on production traffic rather than episodic benchmark runs. Our analysis shows that standard metrics fail to detect four of the seven failure modes entirely and detect three others only after a lag of multiple evaluation cycles.

Related papers

• Mind the Gap: Robustness Risks in PII Detection Systems [6.167318681955322]
  We study how different PII detection paradigms fail under noisy, unstructured, and informal inputs.<n>No single architecture is uniformly reliable across PII categories.<n>Motivated by these findings, we propose a hybrid detection pipeline with a QA-driven feedback loop for iterative risk mitigation.
  arXiv  Detail & Related papers  (2026-09-03T07:23:28Z)
• HarnessEval-W: Agentifying the Evaluation of Visual Worlds [149.63310396189317]
  HarnessEval-W is an agentified evaluation pipeline for benchmarking world models.<n>Rather than applying a fixed rubric, HarnessEval-W decomposes the evaluation question into measurable subproblems.<n>We apply HarnessEval-W to 18 representative world models over 330 evaluation cases.
  arXiv  Detail & Related papers  (2026-08-17T17:43:24Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• Beyond Static Snapshots: A Grounded Evaluation Framework for Language Models at the Agentic Frontier [0.0]
  We present ISOPro, a simulation-based fine-tuning and evaluation system.<n>It replaces the learned reward model with a deterministic ground-truth verifier.<n>It operates on LoRA adapter weights updatable on CPU, reducing the hardware barrier by an order of magnitude.
  arXiv  Detail & Related papers  (2026-04-19T18:28:32Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• ProbeLLM: Automating Principled Diagnosis of LLM Failures [89.44131968886184]
  We propose ProbeLLM, a benchmark-agnostic automated probing framework that elevates weakness discovery from individual failures to structured failure modes.<n>By restricting probing to verifiable test cases and leveraging tool-augmented generation and verification, ProbeLLM grounds failure discovery in reliable evidence.
  arXiv  Detail & Related papers  (2026-02-13T14:33:13Z)
• The Necessity of a Unified Framework for LLM-Based Agent Evaluation [46.631678638677386]
  General-purpose agents have seen fundamental advancements.<n> evaluating these agents presents unique challenges that distinguish them from static QA benchmarks.<n>We propose that a unified evaluation framework is essential for the rigorous advancement of agent evaluation.
  arXiv  Detail & Related papers  (2026-02-03T08:18:37Z)
• How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations [0.0]
  We investigate how large language models (LLMs) fail when operating as autonomous agents with tool-use capabilities.<n>Using the Kamiwaza Agentic Merit Index (KAMI) v0.1 benchmark, we analyze 900 execution traces from three representative models.<n>We identify four recurring failure archetypes: premature action without grounding, over-helpfulness that substitutes missing entities, vulnerability to distractor-induced context pollution, and fragile execution.
  arXiv  Detail & Related papers  (2025-12-08T12:27:15Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models [62.83226685925107]
  Deceptive Reasoning Exposure Suite (D-REX) is a novel dataset designed to evaluate the discrepancy between a model's internal reasoning process and its final output.<n>Each sample in D-REX contains the adversarial system prompt, an end-user's test query, the model's seemingly innocuous response, and, crucially, the model's internal chain-of-thought.<n>We demonstrate that D-REX presents a significant challenge for existing models and safety mechanisms.
  arXiv  Detail & Related papers  (2025-09-22T15:59:40Z)
• BELLS: A Framework Towards Future Proof Benchmarks for the Evaluation of LLM Safeguards [43.86118338226387]
  We introduce the Benchmarks for the Evaluation of LLM Safeguards (BELLS) BELLS is a structured collection of tests, organized into three categories: established failure tests, emerging failure tests and next-gen architecture tests. We implement and share the first next-gen architecture test, using the MACHIAVELLI environment, along with an interactive visualization of the dataset.
  arXiv  Detail & Related papers  (2024-06-03T14:32:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.