Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing

• URL: http://arxiv.org/abs/2507.03870v1
• Date: Sat, 05 Jul 2025 02:50:41 GMT
• Title: Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing
• Abstract summary: We introduce AIProbe, a novel black-box testing technique that applies differential testing to attribute undesirable agent behaviors.<n>AIProbe identifies whether failures are due to errors in the agent's model or policy, or due to unsolvable task conditions.<n>Our evaluation shows that AIProbe significantly outperforms state-of-the-art techniques in detecting both total and unique errors.
• Score: 9.625308787676286
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When an autonomous agent behaves undesirably, including failure to complete a task, it can be difficult to determine whether the behavior is due to a systemic agent error, such as flaws in the model or policy, or an environment error, where a task is inherently infeasible under a given environment configuration, even for an ideal agent. As agents and their environments grow more complex, identifying the error source becomes increasingly difficult but critical for reliable deployment. We introduce AIProbe, a novel black-box testing technique that applies differential testing to attribute undesirable agent behaviors either to agent deficiencies, such as modeling or training flaws, or due to environmental infeasibility. AIProbe first generates diverse environmental configurations and tasks for testing the agent, by modifying configurable parameters using Latin Hypercube sampling. It then solves each generated task using a search-based planner, independent of the agent. By comparing the agent's performance to the planner's solution, AIProbe identifies whether failures are due to errors in the agent's model or policy, or due to unsolvable task conditions. Our evaluation across multiple domains shows that AIProbe significantly outperforms state-of-the-art techniques in detecting both total and unique errors, thereby contributing to a reliable deployment of autonomous agents.

Related papers

• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End? [28.383940617377856]
  We introduce ORAgentBench, an execution-grounded benchmark for evaluating autonomous agents on operations research tasks.<n>It contains 107 human-reviewed tasks across diverse operational scenarios, each packaged with a natural-language brief, multi-file data, configuration artifacts, and a required submission schema.<n>Experiments with fourteen frontier agent-model configurations show that current agents remain far from reliable OR practice.
  arXiv  Detail & Related papers  (2026-06-18T04:43:23Z)
• POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems [0.07783271875179179]
  We present POIROT, a protocol that repurposes a system's own agents as its diagnostic layer.<n>Across evaluated settings, POIROT outperforms single-LLM baseline evaluators.<n>We release POIROT as an open-source library alongside BLAME, a benchmark for fault attribution in safety-critical multi-agent systems.
  arXiv  Detail & Related papers  (2026-06-01T14:05:35Z)
• AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions [78.49000936275773]
  We introduce AgentHijack, a benchmark designed to evaluate the robustness of computer-use agents under common corruptions.<n>We evaluate a variety of desktop tasks that utilize MLLM-based agents and discover that even minor instances of corruption can result in substantial performance degradation.<n>We propose AgentHijack-Agent, a framework that integrates an action generator with enhanced grounding capabilities and an onlooker responsible for behavior summarization and environment checking.
  arXiv  Detail & Related papers  (2026-05-25T11:09:22Z)
• Meta-Agent: From Task Descriptions to Verified Multi-Agent Systems [36.485119937780524]
  We present Meta-Agent, a two-phase framework that automatically constructs and executes specialized multi-agent systems.<n>We introduce a three-level error attribution mechanism that distinguishes local, upstream, and structural failures.<n>Experiments against strong multi-agent baselines and ablation studies demonstrate consistent improvements in task success rate, error recovery, and workflow stability.
  arXiv  Detail & Related papers  (2026-05-24T19:38:54Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments [60.3427704389541]
  Large Language Models are being increasingly deployed as the decision-making core of autonomous agents.<n>Yet, in conversational benchmarks, these agents frequently fail due to the cascading effects of incorrect decision-making.<n>We present the Failure-Aware Meta-Agentic framework to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-28T02:21:53Z)
• Constrained Process Maps for Multi-Agent Generative AI Workflows [10.871587311621974]
  Large language model (LLM)-based agents are increasingly used in regulated settings such as compliance and due diligence.<n>We introduce a multi-agent system formalized as a finite-horizon Markov Decision Process (MDP) with a directed acyclic structure.<n>Epistemic uncertainty is quantified at the agent level using Monte Carlo estimation, while system-level uncertainty is captured by the MDP's termination in either an automated labeled state or a human-review state.
  arXiv  Detail & Related papers  (2026-02-02T12:32:11Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm for Studying Environment Understanding [50.35012849818872]
  Large language model (LLM) agents have demonstrated remarkable capabilities in complex decision-making and tool-use tasks.<n>We propose Task-to-Quiz (T2Q), a deterministic and automated evaluation paradigm designed to decouple task execution from world-state understanding.<n>Our experiments reveal that task success is often a poor proxy for environment understanding, and that current memory machanism can not effectively help agents acquire a grounded model of the environment.
  arXiv  Detail & Related papers  (2026-01-14T14:09:11Z)
• Grounded Test-Time Adaptation for LLM Agents [75.62784644919803]
  Large language model (LLM)-based agents struggle to generalize to novel and complex environments.<n>We propose two strategies for adapting LLM agents by leveraging environment-specific information available during deployment.
  arXiv  Detail & Related papers  (2025-11-06T22:24:35Z)
• Online Robust Multi-Agent Reinforcement Learning under Model Uncertainties [10.054572105379425]
  Well-trained multi-agent systems can fail when deployed in real-world environments.<n>DRMGs enhance system resilience by optimizing for worst-case performance over a defined set of environmental uncertainties.<n>This paper pioneers the study of online learning in DRMGs, where agents learn directly from environmental interactions without prior data.
  arXiv  Detail & Related papers  (2025-08-04T23:14:32Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents.<n>In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches.<n>We show that even moderate levels of information sharing significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)
• Preventing Rogue Agents Improves Multi-Agent Collaboration [21.955058255432974]
  We propose to monitor agents during action prediction and intervene when a future error is likely to occur.<n>Experiments on WhoDunitEnv, code generation tasks and the GovSim environment for resource sustainability show that our approach leads to substantial performance gains.
  arXiv  Detail & Related papers  (2025-02-09T18:35:08Z)
• Defining and Detecting the Defects of the Large Language Model-based Autonomous Agents [31.126001253902416]
  We present the first study focused on identifying and detecting defects in LLM Agents.<n>We collected and analyzed 6,854 relevant posts from StackOverflow to define 8 types of agent defects.<n>Our results show that Agentable achieved an overall accuracy of 88.79% and a recall rate of 91.03%.
  arXiv  Detail & Related papers  (2024-12-24T11:54:14Z)
• Agent-Oriented Planning in Multi-Agent Systems [54.429028104022066]
  We propose AOP, a novel framework for agent-oriented planning in multi-agent systems.<n>In this study, we identify three critical design principles of agent-oriented planning, including solvability, completeness, and non-redundancy.<n> Extensive experiments demonstrate the advancement of AOP in solving real-world problems compared to both single-agent systems and existing planning strategies for multi-agent systems.
  arXiv  Detail & Related papers  (2024-10-03T04:07:51Z)
• Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions [68.92637077909693]
  This paper investigates the faithfulness of multimodal large language model (MLLM) agents in a graphical user interface (GUI) environment.<n>A general scenario is proposed where both the user and the agent are benign, and the environment, while not malicious, contains unrelated content.<n> Experimental results reveal that even the most powerful models, whether generalist agents or specialist GUI agents, are susceptible to distractions.
  arXiv  Detail & Related papers  (2024-08-05T15:16:22Z)
• A Unified Debugging Approach via LLM-Based Multi-Agent Synergy [39.11825182386288]
  FixAgent is an end-to-end framework for unified debug through multi-agent synergy. It significantly outperforms state-of-the-art repair methods, fixing 1.25$times$ to 2.56$times$ bugs on the repo-level benchmark, Defects4J.
  arXiv  Detail & Related papers  (2024-04-26T04:55:35Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Reflexion: Language Agents with Verbal Reinforcement Learning [44.85337947858337]
  Reflexion is a novel framework to reinforce language agents not by updating weights, but through linguistic feedback. It is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals. For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%.
  arXiv  Detail & Related papers  (2023-03-20T18:08:50Z)
• Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design [121.73425076217471]
  We propose Unsupervised Environment Design (UED), where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED) Our experiments demonstrate that PAIRED produces a natural curriculum of increasingly complex environments, and PAIRED agents achieve higher zero-shot transfer performance when tested in highly novel environments.
  arXiv  Detail & Related papers  (2020-12-03T17:37:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.