Robustness Analysis of Agentic AI to Inconsistent and Incomplete Tool Responses

• URL: http://arxiv.org/abs/2608.22676v2
• Date: Sun, 30 Aug 2026 13:25:02 GMT
• Title: Robustness Analysis of Agentic AI to Inconsistent and Incomplete Tool Responses
• Abstract summary: We analyze tool failures at the moment a return enters the agent context.<n>We show that likelihood-based signals clearly capture incomplete returns and some direct inconsistencies.<n>Some failures that are weak under likelihood signals can still redirect the agent toward state-changing actions.
• Score: 8.002494437891077
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool-using agents increasingly rely on external tools to complete multi-step tasks, but tool returns can fail in different ways and require different recovery actions. Existing robustness studies often use uncertainty-based measures to detect when an agent becomes unreliable. These measures can reveal that something has gone wrong, but they do not directly identify the type of tool failure or the appropriate response. We address this limitation by analyzing tool failures at the moment a return enters the agent context. Our approach combines two complementary signals. The first compares the likelihood of the returned content under the tool schema and under the full trajectory prefix. The second measures the agent's probability distribution over its legal next actions. We evaluate the approach by injecting incomplete and inconsistent returns into a retail customer-service benchmark. The results show that likelihood-based signals clearly capture incomplete returns and some direct inconsistencies, while action-based signals reveal how strongly a failure changes the next decision. Some failures that are weak under likelihood signals can still redirect the agent toward state-changing actions. These findings show that tool failures can be recognized at the return boundary, but reliable diagnosis requires combining multiple signals.

Related papers

• Agents Trust Tools Too Much: Measuring Reliance on Unreliable Tools [16.535602696436424]
  We evaluate how agents respond to unreliable tool returns using three tools-web search, LLM sub-agent delegation, and code execution.<n>For each tool, we corrupt its returns and measure whether agents adopt the corrupted content in their final answers.<n>Agents exhibit high levels of overtrust across all three settings.
  arXiv  Detail & Related papers  (2026-09-04T17:02:32Z)
• ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection [61.4148196085256]
  We present ROPE (Routed Origin Policy Enforcement), which is anchored in a structural notion of trust.<n>Our approach admits two provable guarantees: 1) at every step of a trajectory, no value whose only origin is attacker-writable content reaches an origin-guarded parameter, and 2) no rewording of an injection changes an admission decision.
  arXiv  Detail & Related papers  (2026-08-27T01:22:14Z)
• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.69180947909148]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration [24.010665397925333]
  When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer.<n>We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidirectional activation patching.
  arXiv  Detail & Related papers  (2026-08-16T14:47:33Z)
• Grounding Without Corrective Control: Truth-Tracking Profiles for Large Language Models [0.0]
  Recent work suggests that some large language model representations have content or reference.<n>This paper asks what follows from that gap.<n>An output is answerable when discrepancies can affect what a target- and task-specific arrangement produces, accepts, or withdraws.
  arXiv  Detail & Related papers  (2026-08-14T12:30:19Z)
• Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework [49.16055123488827]
  coding agents powered by large language models can now generate code patches directly from bug reports.<n>No mechanism independently verifies whether it truly resolves the reported problem.<n>We propose RETRACE, a training-free post-generation verification framework.
  arXiv  Detail & Related papers  (2026-08-09T22:59:13Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures [0.0]
  Real-world systems exhibit non-atomic behaviors such as timeouts after dispatch, delayed visibility, and partial state updates.<n>These mismatches lead to reliability issues including duplicate actions, task success, and unnecessary tool executions.<n>A lightweight, verification-aware tool wrapper is introduced that augments tool calls with postcondition verification, verify-before-retry logic, and idempotency keys.
  arXiv  Detail & Related papers  (2026-07-31T16:16:14Z)
• Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories [13.06173852232351]
  Current evaluation metrics reduce such a trajectory to a binary success flag or compare it against a reference by exact matching.<n>We instantiate trajectory evaluation as a distance between the agent's execution graph and a set of valid solution graphs.<n>The resulting score, termed otap, is a pseudo-metric that is provably invariant to dependency-preserving reorderings.
  arXiv  Detail & Related papers  (2026-07-19T05:23:17Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Toward Semantically-Seeded, Graph-Propagated Impact Analysis Across Software Artifacts: A Vision [0.0]
  We argue for a training-free and interpretable analyzer that fuses both signals over the same embeddings.<n>A small but complete proof-of-concept on a payment subsystem labelled change scenarios shows the mechanism we care about.
  arXiv  Detail & Related papers  (2026-06-17T09:34:56Z)
• ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling [57.42714978834704]
  ExComm is a communication protocol for exploration-stage agentic test-time scaling.<n>We show that ExComm consistently outperforms strong test-time scaling baselines.
  arXiv  Detail & Related papers  (2026-05-21T07:38:44Z)
• Beyond the Black Box: Interpretability of Agentic AI Tool Use [0.0]
  We introduce a mechanistic-interpretability toolkit built on Sparse Autoencoders and linear probes.<n>The framework reads model states before each action and infers both whether a tool is needed and how consequential the next tool action is likely to be.<n>We train the probes on multi-step trajectories from the NVIDIA Nemotron function-calling dataset and apply the same workflow to GPT-OSS 20B and Gemma 3 27B models.
  arXiv  Detail & Related papers  (2026-05-07T19:47:30Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• RAGEN-2: Reasoning Collapse in Agentic RL [123.40730283427136]
  In RAGEN-2, we find that even with stable entropy, models can rely on fixed templates that look diverse but are input-agnostic.<n>We decompose reasoning quality into within-input diversity (Entropy) and cross-input distinguishability (Mutual Information, MI)<n>Across diverse tasks, mutual information correlates with final performance much more strongly than entropy, making it a more reliable proxy for reasoning quality.
  arXiv  Detail & Related papers  (2026-04-07T04:29:41Z)
• Willful Disobedience: Automatically Detecting Failures in Agentic Traces [2.2365938060262356]
  AgentPex is an AI-powered tool designed to systematically evaluate agentic traces.<n>It extracts behavioral rules from agent prompts and system instructions, then uses these specifications to automatically evaluate traces for compliance.<n>We evaluate AgentPex on 424 traces from 2-bench across models in telecom, retail, and airline customer service.
  arXiv  Detail & Related papers  (2026-03-25T00:33:39Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• LLM Self-Explanations Fail Semantic Invariance [27.126691338850254]
  A faithful self-report should remain stable when only the semantic context changes while the functional state stays fixed.<n>We operationalize this test in an agentic setting where four frontier models face a deliberately impossible task.
  arXiv  Detail & Related papers  (2026-03-01T20:18:43Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understanding [40.78154629252038]
  Forcing binary labels upon ambiguous claims lowers the reliability of evaluation.<n>We introduce LLM-generated edits of summaries as a method of providing a nuanced evaluation of claims.<n>We show that ARM produces a absolute 21% improvement in annotator agreement on claim faithfulness.
  arXiv  Detail & Related papers  (2025-04-01T19:08:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.