Beyond Component Testing: Validating Agentic AI Systems

• URL: http://arxiv.org/abs/2607.29405v1
• Date: Fri, 31 Jul 2026 13:24:34 GMT
• Title: Beyond Component Testing: Validating Agentic AI Systems
• Abstract summary: Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation.<n>This survey synthesizes 257 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance.
• Score: 2.403279740172383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input--output evaluation, because acceptable system behavior now depends on how decisions unfold over time and under changing environmental conditions. This survey synthesizes 257 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance in order to characterize the validation problem for agentic systems. The review is organized around a five-dimension taxonomy covering behavioral, safety, temporal, regulatory, and multi-agent concerns, and uses that taxonomy to map current approaches and expose recurrent coverage gaps. The analysis shows that behavioral evaluation is comparatively mature, while temporal validity, runtime evidence maintenance, regulatory legibility, and open-ended multi-agent systems assurance remain under-developed. Three cross-domain case studies (medical care, industrial operations, smart-mobility systems) provide operational illustrations of how the five taxonomy dimensions recur in safety-critical settings, grounded in the failure patterns documented in the reviewed literature. The paper concludes with a lifecycle-oriented research agenda centered on bounded-autonomy specifications, adversarial trajectory generation, runtime monitoring, and audit-ready evidence structures. The central claim is that trustworthy deployment of agentic AI depends on validating trajectories in context rather than assessing isolated components alone.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Terminal Agents: A Survey of AI Agents in Command-Line Environments [102.48222742145474]
  Large language model agents increasingly act through terminals, yet existing surveys disperse terminal-mediated behavior across software engineering, tool use, and computer-use research.<n>We regard terminal agents as systems whose dominant progress-bearing action--observation loop is mediated by terminal command execution, textual feedback, and stateful environment interaction.<n>Our synthesis shows that realized behavior is jointly shaped by the model, interface, harness, runtime, and environment.
  arXiv  Detail & Related papers  (2026-08-20T18:16:44Z)
• Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model [0.0]
  Book synthesizes 164 scholarly works, 100 practitioner records, 29 benchmark records, and 17 author-system case records.<n>Written as a structured multivocal review, targeted update audits, software-engineering coverage analysis, and distributed-systems evidence synthesis.<n>Book contributes a versioned catalog of 206 reliability records, including 56 developed in depth, plus 13 research leads.
  arXiv  Detail & Related papers  (2026-08-14T01:34:25Z)
• DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations [106.74348504930124]
  We introduce DocOps, a deterministically verifiable evaluation framework underpinned by a hierarchical taxonomy.<n>Based on DocOps, we systematically evaluate representative closed- and open-source models across various agentic harnesses.<n>Our work exposes the capability boundaries of agents in maintaining global document consistency, shedding light on the future design of robust, non-destructive agents.
  arXiv  Detail & Related papers  (2026-07-22T07:52:59Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• Referential Security as a New Paradigm for AI Evaluations [5.748110232428846]
  We propose referential security as a new paradigm for AI evaluation.<n>This approach reframes model identity as an empirically verifiable property and separates referential stability from the substantive security claims it conditions.
  arXiv  Detail & Related papers  (2026-05-25T10:24:10Z)
• Monitoring Data-aware Temporal Properties (Extended Version) [56.386411908764494]
  We consider anticipatory monitoring of linear-time properties enriched with an arbitrary SMT theory over finite traces.<n>Anticipatory monitoring in this setting is highly challenging, as the monitoring state depends on both the trace prefix seen so far and all its possible finite continuations.<n>We present and formally prove the correctness of a novel foundational framework for monitoring properties in an expressive fragment offMT.
  arXiv  Detail & Related papers  (2026-05-14T10:23:11Z)
• Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
  We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
  arXiv  Detail & Related papers  (2026-05-10T06:30:19Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI [0.0]
  This paper introduces a novel auditing framework to quantify latent trait estimation under ordinal uncertainty.<n>The research audits nine leading models across dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization.
  arXiv  Detail & Related papers  (2026-02-19T06:56:01Z)
• The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis [24.51410516475904]
  This SoK presents a comprehensive overview of the Prompt Injection (PI) landscape, covering attacks, defenses, and their evaluation practices.<n>We introduce AgentPI, a new benchmark designed to systematically evaluate agent behavior under context-dependent interaction settings.<n>We show that many defenses appear effective under existing benchmarks by suppressing contextual inputs, yet fail to generalize to realistic agent settings where context-dependent reasoning is essential.
  arXiv  Detail & Related papers  (2026-02-11T02:47:10Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems [0.0]
  Recent advances in agentic AI have shifted the focus from standalone Large Language Models to integrated systems.<n>We propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment.<n>We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations by conventional metrics.
  arXiv  Detail & Related papers  (2025-12-14T18:17:40Z)
• AgentGuard: Runtime Verification of AI Agents [1.14219428942199]
  AgentGuard is a framework for runtime verification of Agentic AI systems.<n>It provides continuous, quantitative assurance through a new paradigm called Dynamic Probabilistic Assurance.
  arXiv  Detail & Related papers  (2025-09-28T13:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.