Mahalanobis-Guided Latent OOD Detection for Hybrid ES-DRL Control in Time-Varying Systems

• URL: http://arxiv.org/abs/2606.11474v2
• Date: Sat, 13 Jun 2026 20:34:39 GMT
• Title: Mahalanobis-Guided Latent OOD Detection for Hybrid ES-DRL Control in Time-Varying Systems
• Abstract summary: We study Mahalanobis-guided latent out-of-distribution (OOD) detection for test-time RL controller switching in nonlinear time-varying systems.<n>We consider a combined ES--DRL controller, where RL provides fast in-distribution actions and bounded extremum seeking (ES) provides robust model-independent control under OOD operation.<n>We train a variational autoencoder (VAE) on in-distribution beam-profile observations and use Mahalanobis distance in the VAE latent space to detect OOD beam profiles at test time.
• Score: 45.88028371034407
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we study Mahalanobis-guided latent out-of-distribution (OOD) detection for test-time RL controller switching in nonlinear time-varying systems. RL controllers can quickly control high-dimensional systems within the training distribution, but their performance can degrade when time-varying dynamics produce unseen observations. We consider a combined ES--DRL controller, where RL provides fast in-distribution actions and bounded extremum seeking (ES) provides robust model-independent control under OOD operation. The key challenge is deciding when to switch. We train a variational autoencoder (VAE) on in-distribution beam-profile observations and use Mahalanobis distance in the VAE latent space to detect OOD beam profiles at test time. This OOD decision sets a binary switch that selects either the RL controller or the ES controller. We evaluate the approach in safety-critical particle accelerator control. In this setting, spatial magnet motion creates OOD beam profiles that were not seen during RL training. Visualization of the VAE latent space shows that the proposed method identifies this OOD scenario and provides an interpretable signal for switching between RL and ES in the combined controller.

Related papers

• Hybrid Neural Network and Conventional Controller Approach for Robust Control of Highly Unstable Systems: Application to Tilt-Rotor Control [0.5599792629509229]
  Multirotors are widely used in applications ranging from surveillance to precision agriculture, yet conventional designs remain limited by their under-actuation.<n>This paper investigates neural-network-based control strategies for a fully actuated tiltrotor system with four thrust-vectoring inputs.
  arXiv  Detail & Related papers  (2026-06-07T16:10:46Z)
• Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning [42.15812524999639]
  Agentic Fast-Slow Planning is a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales.<n>We show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline.
  arXiv  Detail & Related papers  (2026-04-02T06:34:29Z)
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning [88.42566960813438]
  CalibRL is a hybrid-policy RLVR framework that supports controllable exploration with expert guidance.<n>CalibRL increases policy entropy in a guided manner and clarifies the target distribution.<n>Experiments across eight benchmarks, including both in-domain and out-of-domain settings, demonstrate consistent improvements.
  arXiv  Detail & Related papers  (2026-02-22T07:23:36Z)
• Control of a Twin Rotor using Twin Delayed Deep Deterministic Policy Gradient (TD3) [0.0]
  This paper proposes a reinforcement learning framework for stabilizing the Twin Rotor Aerodynamic System (TRAS)<n>The complex dynamics and non-linear characteristics of the TRAS make it challenging to control using traditional control algorithms.<n>Experiments on a laboratory setup were carried out to confirm the controller's effectiveness in real-world applications.
  arXiv  Detail & Related papers  (2025-12-15T14:10:04Z)
• Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seeking [39.407739937584104]
  We study the use of robust model independent bounded extremum seeking (ES) feedback control to improve the robustness of deep reinforcement learning controllers.<n>We present a numerical study of a general time-varying system and a combined ES-DRL controller for automatic tuning of the Low Energy Beam Transport section at the Los Alamos Neutron Science Center linear particle accelerator.
  arXiv  Detail & Related papers  (2025-10-02T18:53:02Z)
• ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomy [51.56484100374058]
  ASTREA is the first agentic system executed on flight-heritage hardware for autonomous spacecraft operations.<n>We integrate a resource-constrained Large Language Model (LLM) agent with a reinforcement learning controller in an asynchronous architecture tailored for space-qualified platforms.
  arXiv  Detail & Related papers  (2025-09-16T08:52:13Z)
• Blackout Mitigation via Physics-guided RL [17.807967857394406]
  This paper considers the sequential design of remedial control actions in response to system anomalies for the ultimate objective of preventing blackouts. A physics-guided reinforcement learning framework is designed to identify effective sequences of real-time remedial look-ahead decisions.
  arXiv  Detail & Related papers  (2024-01-17T23:27:36Z)
• AUTO: Adaptive Outlier Optimization for Test-Time OOD Detection [79.51071170042972]
  Out-of-distribution (OOD) detection aims to detect test samples that do not fall into any training in-distribution (ID) classes.<n>Data safety and privacy make it infeasible to collect task-specific outliers in advance for different scenarios.<n>We present test-time OOD detection, which allows the deployed model to utilize real OOD data from the unlabeled data stream during testing.
  arXiv  Detail & Related papers  (2023-03-22T02:28:54Z)
• Time-to-Green predictions for fully-actuated signal control systems with supervised learning [56.66331540599836]
  This paper proposes a time series prediction framework using aggregated traffic signal and loop detector data. We utilize state-of-the-art machine learning models to predict future signal phases' duration. Results based on an empirical data set from a fully-actuated signal control system in Zurich, Switzerland, show that machine learning models outperform conventional prediction methods.
  arXiv  Detail & Related papers  (2022-08-24T07:50:43Z)
• Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results [21.054448068345348]
  We study the problem of out-of-distribution dynamics (OODD) detection, which involves detecting when the dynamics of a temporal process change compared to the training-distribution dynamics. This problem is particularly important in the context of deep RL, where learned controllers often overfit to the training environment. Our first contribution is to design a set of OODD benchmarks derived from common RL environments with varying types and intensities of OODD. Our second contribution is to design a strong OODD baseline approach based on recurrent implicit quantile networks (RIQNs), which monitors autoregressive prediction errors for OODD detection.
  arXiv  Detail & Related papers  (2021-07-11T06:40:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.