Failure-Based Testing for Deep Reinforcement Learning Agents

• URL: http://arxiv.org/abs/2606.31372v1
• Date: Tue, 30 Jun 2026 09:05:29 GMT
• Title: Failure-Based Testing for Deep Reinforcement Learning Agents
• Abstract summary: Prior Random Testing is a failure-based testing method for Deep Reinforcement Learning (DRL) agents.<n> PRT prioritizes failure-prone regions, thereby facilitating efficient failure detection.<n> PRT ranks among the top performers in terms of both the cost of finding the first failure and the diversity of test cases.
• Score: 1.8397573213627283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Reinforcement Learning (DRL) agents have been widely adopted across diverse domains to address challenging decision-making problems, such as autonomous driving and robotic control. Given that many of these applications are safety- and security-critical, rigorous testing of DRL agents is indispensable. Existing testing methods are typically guided by reward signals to detect failures. However, for well-trained agents, whose performance approaches optimal levels in standard operating conditions, reward signals remain generally high, making current methods ineffective at uncovering critical failures. To address these challenges, we propose a novel failure-based method that leverages task-induced failure insights to enhance failure detection capability while reducing the number of tests required. Since DRL agents are inherently designed with human-defined tasks, they provide valuable cues about task difficulty. Intuitively, a DRL agent is more likely to fail when confronted with a more difficult task; therefore, PRT prioritizes these tasks. Building on this foundation, we propose Prior Random Testing, a black-box failure-based testing method that enables targeted prioritization while preserving the diversity of generated test cases. Guided by task-induced failure insights, PRT prioritizes failure-prone regions of the input domain, thereby facilitating efficient failure detection. PRT is evaluated on four widely used benchmarks and compared with different state-of-the-art methods including fuzzing, search-based and generative-based methods. PRT ranks among the top performers in terms of both the cost of finding the first failure and the diversity of test cases. Notably, compared to random testing, PRT achieves better diversity and reduces the testing cost by over 50%.

Related papers

• Learning from the Test: Self-Referential Differential Testing for Deep RL Agents [10.95548749552727]
  Deep Reinforcement Learning (DRL) has achieved significant success in complex decision-making problems.<n>Current works primarily focus on detecting safety-critical failures, often neglecting policy optimality.<n>We propose Delta, a framework that automatically identifies both safety-critical and optimality bugs in DRL agents.
  arXiv  Detail & Related papers  (2026-08-23T08:22:27Z)
• FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents [49.519184792774304]
  FailForge is an agentic framework that converts failed rollouts into training signal.<n>FailForge recovers over 26% of previously failed instances at marginal additional cost.
  arXiv  Detail & Related papers  (2026-08-09T08:22:57Z)
• Verified Critical Step Optimization for LLM Agents [67.05296684575445]
  Critical Step Optimization focuses preference learning on verified critical steps.<n>Method starts from failed policy trajectories rather than expert demonstrations.<n>Experiments on GAIA-Text-103 and XBench-DeepSearch show that CSO achieves 37% and 26% relative improvement over the SFT baseline.
  arXiv  Detail & Related papers  (2026-02-03T11:41:02Z)
• BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search [72.87861928940929]
  Boundary-Aware Policy Optimization (BAPO) is a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy.<n>BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut.
  arXiv  Detail & Related papers  (2026-01-16T07:06:58Z)
• An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications [12.166151903597445]
  Foundation model (FM)-based AI agents are rapidly gaining adoption across diverse domains.<n>Their inherent non-determinism and non-reproducibility pose testing and quality assurance challenges.<n>We conduct the first large-scale empirical study of testing practices in the AI agent ecosystem.
  arXiv  Detail & Related papers  (2025-09-23T16:02:09Z)
• Exploring Expert Failures Improves LLM Agent Tuning [74.0772570556016]
  We propose Exploring Expert Failures (EEF), which identifies beneficial actions from failed expert trajectories.<n>EEF successfully solves some previously unsolvable subtasks and improves agent tuning performance.
  arXiv  Detail & Related papers  (2025-04-17T17:53:54Z)
• Testing of Deep Reinforcement Learning Agents with Surrogate Models [10.243488468625786]
  Deep Reinforcement Learning (DRL) has received a lot of attention from the research community in recent years. In this paper, we propose a search-based approach to test such agents.
  arXiv  Detail & Related papers  (2023-05-22T06:21:39Z)
• A Search-Based Testing Approach for Deep Reinforcement Learning Agents [1.1580916951856255]
  We propose a Search-based Testing Approach of Reinforcement Learning Agents (STARLA) to test the policy of a DRL agent. We use machine learning models and a dedicated genetic algorithm to narrow the search towards faulty episodes.
  arXiv  Detail & Related papers  (2022-06-15T20:51:33Z)
• GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning [2.3309136820332164]
  We introduce expert knowledge to guide the agent to make better decisions in RL-based PT. We propose a Generative Adversarial Imitation Learning-based generic intelligent Penetration testing framework, denoted as GAIL-PT.
  arXiv  Detail & Related papers  (2022-04-05T04:01:17Z)
• Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
  Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs. We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2020-08-05T07:49:42Z)
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations [88.94162416324505]
  A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises. Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions. We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
  arXiv  Detail & Related papers  (2020-03-19T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.