OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning

• URL: http://arxiv.org/abs/2607.12523v1
• Date: Tue, 14 Jul 2026 08:57:04 GMT
• Title: OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning
• Abstract summary: We present OOD-RL-Bench, a framework designed to evaluate OOD detectors against categories of anomalies injected into trajectories.<n> Detectors and anomaly injectors are integrated through shared interfaces and configuration.<n>We assess the performance of each detector across a suite of anomaly types using matched-time AUROC, matched-time AUPRC, matched-time false-positive rate, detection delay, and segmented-onset metrics.
• Score: 4.173831365316023
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable reinforcement learning (RL) agents must maintain operational integrity amidst sensor malfunctions, dynamic disturbances, and slow environmental shifts. The detection of out-of-distribution conditions is pivotal to determining when an agent's observations, transitions, or trajectory dynamics deviate from the assumptions underpinning its policy training. Current out-of-distribution (OOD) detection benchmarks typically evaluate image classifiers or static low-dimensional datasets, failing to account for the complex, action-dependent temporal structure inherent in RL trajectories. To address this gap, we present OOD-RL-Bench, a comprehensive and extensible framework designed to evaluate OOD detectors against categories of anomalies injected into RL trajectories. Detectors and anomaly injectors are integrated through shared interfaces and configuration, which allows new scoring methods and perturbation families to be evaluated without modification of the core benchmark loop. We evaluate the utility of the framework using a Deep Q-Network policy within the LunarLander-v3 environment. We assess the performance of each detector across a suite of anomaly types using matched-time AUROC, matched-time AUPRC, matched-time false-positive rate, detection delay, and segmented-onset metrics. Our analysis reveals significant performance variance across anomaly types: observation perturbations and regime switches are identified with high accuracy by several methods, while observation delay and action-conditioned dynamics remain difficult even when post-onset anomaly scores are compared against clean scores from the same timesteps. We make the framework, trained policy checkpoint, and complete results publicly available as a reproducible artefact.

Related papers

• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• TimeLAVA: Learning-Agnostic Data Valuation for Time Series [48.76711740818533]
  TimeLAVA is a learning-agnostic framework that values temporal segments by their marginal contribution to minimize distributional discrepancy between evaluated and reference data.<n>We provide theoretical guarantees linking valuation to model-agnostic generalization and prove bounded sensitivity to contamination.
  arXiv  Detail & Related papers  (2026-06-17T06:16:16Z)
• Disjoint or Overlapping? Inference Windowing for Reconstruction-Based Time Series Anomaly Detection [1.5439079597766574]
  We study the role of the inference stride, which controls whether subsequences are processed as disjoint windows or with overlap.<n>We propose a unified training, tuning, and multi-seed evaluation protocol on the curated TSB-AD benchmark.<n>We show that overlapping windows yield consistent improvements, with average relative gain up to +28%, and can alter method rankings.
  arXiv  Detail & Related papers  (2026-06-02T19:42:30Z)
• Conditional Attribution for Root Cause Analysis in Time-Series Anomaly Detection [13.555771273974697]
  We propose a conditional attribution framework that explains anomalies relative to contextually similar normal system states.<n>Our method retrieves representative normal instances conditioned on the anomalous observation, enabling dependency-preserving and operationally meaningful explanations.<n> Experiments on the SWaT and benchmarks demonstrate that the proposed approach consistently improves root-cause identification accuracy, temporal localization, and robustness across multiple anomaly detection models.
  arXiv  Detail & Related papers  (2026-04-19T21:01:52Z)
• Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation [3.43058724483837]
  Time series cross-validation (TSCV) techniques aim to preserve temporal ordering during model evaluation.<n>This study systematically investigates the effect of TSCV strategy on the precision-recall characteristics of classifiers trained to detect fault-like anomalies in MTS datasets.
  arXiv  Detail & Related papers  (2025-06-13T19:14:44Z)
• Adaptive Thresholding Heuristic for KPI Anomaly Detection [1.57731592348751]
  A plethora of outlier detectors have been explored in the time series domain, however, in a business sense, not all outliers are anomalies of interest. This article proposes an Adaptive Thresholding Heuristic (ATH) to dynamically adjust the detection threshold based on the local properties of the data distribution and adapt to changes in time series patterns. Experimental results show that ATH is efficient making it scalable for near real time anomaly detection and flexible with forecasters and outlier detectors.
  arXiv  Detail & Related papers  (2023-08-21T06:45:28Z)
• DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization [58.704753031608625]
  Time series is one of the most challenging modalities in machine learning research. OOD detection and generalization on time series tend to suffer due to its non-stationary property. We propose DIVERSIFY, a framework for OOD detection and generalization on dynamic distributions of time series.
  arXiv  Detail & Related papers  (2023-08-04T12:27:11Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.