Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study

• URL: http://arxiv.org/abs/2602.14322v2
• Date: Sun, 22 Feb 2026 22:08:26 GMT
• Title: Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study
• Abstract summary: We investigate how formal temporal logic specifications can enhance the safety and robustness of reinforcement learning control in aerospace applications.<n>Experiments show that the conformal shield preserves STL satisfaction while maintaining near baseline performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate how formal temporal logic specifications can enhance the safety and robustness of reinforcement learning (RL) control in aerospace applications. Using the open source AeroBench F-16 simulation benchmark, we train a Proximal Policy Optimization (PPO) agent to regulate engine throttle and track commanded airspeed. The control objective is encoded as a Signal Temporal Logic (STL) requirement to maintain airspeed within a prescribed band during the final seconds of each maneuver. To enforce this specification at run time, we introduce a conformal STL shield that filters the RL agent's actions using online conformal prediction. We compare three settings: (i) PPO baseline, (ii) PPO with a classical rule-based STL shield, and (iii) PPO with the proposed conformal shield, under both nominal conditions and a severe stress scenario involving aerodynamic model mismatch, actuator rate limits, measurement noise, and mid-episode setpoint jumps. Experiments show that the conformal shield preserves STL satisfaction while maintaining near baseline performance and providing stronger robustness guarantees than the classical shield. These results demonstrate that combining formal specification monitoring with data driven RL control can substantially improve the reliability of autonomous flight control in challenging environments.

Related papers

• PhaseLoRA: Control-Regime-Conditioned Low-Rank Adaptation for Continuous-Action Vision-Language-Action Policies [74.07190331264488]
  We propose a lightweight LoRA parameterization that conditions adaptation at each action-chunk prediction step.<n>PhaseLoRA modulates the LoRA left factor in the action expert, allowing the effective low-rank update direction to vary over time.<n>Results establish within-trajectory conditioning as an effective lightweight PEFT axis for continuous-action VLA policies.
  arXiv  Detail & Related papers  (2026-08-15T15:32:08Z)
• AI-Augmented Model Predictive Control for Safe and Adaptive Rendezvous and Proximity Operations [2.787067333542931]
  This work presents an adaptive Model Predictive Control (MPC) framework for autonomous spacecraft RPO in multi-agent adversarial scenarios.<n>The framework was evaluated in the official Kerbal Space Program Differential Game (KSPDG) Capture-the-Satellite environment through Monte Carlo simulations.
  arXiv  Detail & Related papers  (2026-07-18T04:20:23Z)
• Joint Air Traffic Flow and Capacity Management via Answer Set Programming [64.47906587579985]
  We introduce a joint ATFCM model with an encoding in Answer Set Programming (ASP)<n>Our results indicate that the ASP model outperforms the MIP model.<n>While DAC has the largest improvement on solving performance compared to rerouting, unrestricted variants of DAC or rerouting lead to search space thrashing.
  arXiv  Detail & Related papers  (2026-06-22T07:58:54Z)
• Memory-Efficient Meta-Reinforcement Learning for Adaptive Safety-Critical Control in Adversarial Spacecraft Proximity Operations [42.47287159687486]
  This paper investigates the performance of three recurrent network architectures (LSTM, Gated Recurrent Unit (GRU), Selective State Space Model (Mamba)) and two training algorithms (Proximal Policy Optimization (PPO) and Soft Actor Critic (SAC)) to identify the best setup for tuning ICCBF class-K functions via meta-RL.<n>Results indicate that state space models such as Mamba when used with PPO achieve superior task completion, safety, and fuel-savings compared to other architectures.
  arXiv  Detail & Related papers  (2026-06-16T01:53:54Z)
• A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions [14.576960188691215]
  Reinforcement learning (RL)-based quadrotor control policies have achieved impressive performance in tasks such as fast navigation in cluttered environments and drone racing.<n>We present a novel approach to achieve tunable performance in RL-based Quadrotor control through reward design and termination conditions.
  arXiv  Detail & Related papers  (2026-05-18T22:40:30Z)
• Hypernetwork-Conditioned Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Actuator Failures [0.8164433158925594]
  This paper presents a reinforcement learning-based path-following controller for a small uncrewed aircraft system.<n>We consider parameter-efficient formulations based on Feature-wise Linear Modulation (FiLM) and Low-Rank Adaptation (LoRA)<n>We demonstrate that hypernetwork-conditioned policies can improve robustness compared to standard multilayer perceptron policies.
  arXiv  Detail & Related papers  (2026-04-03T18:50:31Z)
• Control of a Twin Rotor using Twin Delayed Deep Deterministic Policy Gradient (TD3) [0.0]
  This paper proposes a reinforcement learning framework for stabilizing the Twin Rotor Aerodynamic System (TRAS)<n>The complex dynamics and non-linear characteristics of the TRAS make it challenging to control using traditional control algorithms.<n>Experiments on a laboratory setup were carried out to confirm the controller's effectiveness in real-world applications.
  arXiv  Detail & Related papers  (2025-12-15T14:10:04Z)
• Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learning [46.90899478779653]
  Shielding is widely used to enforce safety in reinforcement learning (RL)<n>We develop the first adaptive shielding framework based on Generalized Reactivity of rank 1 (GR(1)) specifications.<n>Our method detects environment assumption violations at runtime and employs Inductive Logic Programming (ILP) to automatically repair GR(1) specifications online.
  arXiv  Detail & Related papers  (2025-11-04T14:27:28Z)
• ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomy [51.56484100374058]
  ASTREA is the first agentic system executed on flight-heritage hardware for autonomous spacecraft operations.<n>We integrate a resource-constrained Large Language Model (LLM) agent with a reinforcement learning controller in an asynchronous architecture tailored for space-qualified platforms.
  arXiv  Detail & Related papers  (2025-09-16T08:52:13Z)
• Real Time Control of Tandem-Wing Experimental Platform Using Concerto Reinforcement Learning [0.0]
  This paper introduces the CRL2RT algorithm, an advanced reinforcement learning method aimed at improving the real-time control performance of the Direct-Drive Tandem-Wing Experimental Platform (DDTWEP)<n>Results show that CRL2RT achieves a control frequency surpassing 2500 Hz on standard CPUs.
  arXiv  Detail & Related papers  (2025-02-08T03:46:40Z)
• Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters [0.0]
  Reinforcement learning is increasingly used to adapt to complex scenarios, but standard frameworks ensuring safety and stability are lacking. Predictive Safety Filters (PSF) offer a promising solution, ensuring constraint satisfaction in learning-based control without explicit constraint handling. We apply this approach to marine navigation, combining RL with PSF on a simulated Cybership II model. Results demonstrate the PSF's effectiveness in maintaining safety without hindering the RL agent's learning rate and performance, evaluated against a standard RL agent without PSF.
  arXiv  Detail & Related papers  (2023-12-04T12:37:54Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• Signal Temporal Logic Neural Predictive Control [15.540490027770621]
  We propose a method to learn a neural network controller to satisfy the requirements specified in Signal temporal logic (STL) Our controller learns to roll out trajectories to maximize the STL robustness score in training. A backup policy is designed to ensure safety when our controller fails.
  arXiv  Detail & Related papers  (2023-09-10T20:31:25Z)
• Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference [73.63024765499719]
  Mobile networks can be compromised by interference from neighboring cells or jammers. We propose Spectrum Breathing, which cascades-gradient pruning and spread spectrum to suppress interference without bandwidth expansion. We show a performance tradeoff between gradient-pruning and interference-induced error as regulated by the breathing depth.
  arXiv  Detail & Related papers  (2023-05-10T07:05:43Z)
• Designing a Robust Low-Level Agnostic Controller for a Quadrotor with Actor-Critic Reinforcement Learning [0.38073142980732994]
  We introduce domain randomization during the training phase of a low-level waypoint guidance controller based on Soft Actor-Critic. We show that, by introducing a certain degree of uncertainty in quadrotor dynamics during training, we can obtain a controller that is capable to perform the proposed task using a larger variation of quadrotor parameters.
  arXiv  Detail & Related papers  (2022-10-06T14:58:19Z)
• Time-to-Green predictions for fully-actuated signal control systems with supervised learning [56.66331540599836]
  This paper proposes a time series prediction framework using aggregated traffic signal and loop detector data. We utilize state-of-the-art machine learning models to predict future signal phases' duration. Results based on an empirical data set from a fully-actuated signal control system in Zurich, Switzerland, show that machine learning models outperform conventional prediction methods.
  arXiv  Detail & Related papers  (2022-08-24T07:50:43Z)
• Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty [77.18483084440182]
  Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. We present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs.
  arXiv  Detail & Related papers  (2021-06-13T23:08:49Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.