TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios

• URL: http://arxiv.org/abs/2602.13040v1
• Date: Fri, 13 Feb 2026 15:48:20 GMT
• Title: TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios
• Abstract summary: Constrained Reinforcement Learning aims to optimize decision-making policies under constraint conditions.<n>Existing robust CRL approaches predominantly focus on single-step perturbations and temporally independent adversarial models.<n>We propose temporal-coupled adversarial training framework for robust constrained reinforcement learning (TCRL) in worst-case scenarios.
• Score: 7.3881187966342905
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Constrained Reinforcement Learning (CRL) aims to optimize decision-making policies under constraint conditions, making it highly applicable to safety-critical domains such as autonomous driving, robotics, and power grid management. However, existing robust CRL approaches predominantly focus on single-step perturbations and temporally independent adversarial models, lacking explicit modeling of robustness against temporally coupled perturbations. To tackle these challenges, we propose TCRL, a novel temporal-coupled adversarial training framework for robust constrained reinforcement learning (TCRL) in worst-case scenarios. First, TCRL introduces a worst-case-perceived cost constraint function that estimates safety costs under temporally coupled perturbations without the need to explicitly model adversarial attackers. Second, TCRL establishes a dual-constraint defense mechanism on the reward to counter temporally coupled adversaries while maintaining reward unpredictability. Experimental results demonstrate that TCRL consistently outperforms existing methods in terms of robustness against temporally coupled perturbation attacks across a variety of CRL tasks.

Related papers

• Survival Reinforcement Learning: Toward Scalable Self-Supervised RL [10.768601219140153]
  Self-supervised Contrastive Reinforcement Learning (CRL) has shown remarkable depth-scaling capabilities.<n>We introduce Survival Reinforcement Learning (SRL), an online classification-based alternative that extends the survival value learning framework.
  arXiv  Detail & Related papers  (2026-05-29T13:05:08Z)
• Conditional Sequence Modeling for Safe Reinforcement Learning [8.858563919623082]
  offline safe reinforcement learning aims to learn policies from a fixed dataset while maximizing performance under cumulative cost constraints.<n>Most existing offline safe RL methods are trained under a pre-specified threshold.<n>We propose RCDT, a CSM-based method that supports zero-shot deployment across multiple cost thresholds within a single trained policy.
  arXiv  Detail & Related papers  (2026-02-09T12:22:57Z)
• Sparse Threats, Focused Defense: Criticality-Aware Robust Reinforcement Learning for Safe Autonomous Driving [11.62520853262219]
  We introduce criticality-aware robust RL (CARRL) for handling sparse, safety-critical risks in autonomous driving.<n>CARRL consists of two interacting components: a risk exposure adversary (REA) and a risk-targeted robust agent (RTRA)<n>We show that our approach reduces the collision rate by at least 22.66% across all cases compared to state-of-the-art baseline methods.
  arXiv  Detail & Related papers  (2026-01-05T05:20:16Z)
• Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts [68.18666621908898]
  Single-domain offline reinforcement learning (RL) often suffers from limited data coverage.<n>Cross-domain offline RL handles this issue by leveraging additional data from other domains with dynamics shifts.<n>In this paper, we investigate dual (both train-time and test-time) robustness against dynamics shifts in cross-domain offline RL.
  arXiv  Detail & Related papers  (2025-12-02T07:20:39Z)
• Large Reasoning Models Learn Better Alignment from Flawed Thinking [56.08883934423522]
  Large reasoning models (LRMs) "think" by generating structured chain-of-thought (CoT) before producing a final answer.<n>We propose RECAP, a principled reinforcement learning (RL) method for post-training that explicitly teaches models to override flawed reasoning trajectories.
  arXiv  Detail & Related papers  (2025-10-01T14:15:43Z)
• Distributionally Robust Constrained Reinforcement Learning under Strong Duality [37.76993170360821]
  We study the problem of Distributionally Robust Constrained RL (DRC-RL) The goal is to maximize the expected reward subject to environmental distribution shifts and constraints. We develop an algorithmic framework based on strong duality that enables the first efficient and provable solution.
  arXiv  Detail & Related papers  (2024-06-22T08:51:57Z)
• Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations [98.5802673062712]
  We introduce temporally-coupled perturbations, presenting a novel challenge for existing robust reinforcement learning methods. We propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game.
  arXiv  Detail & Related papers  (2023-07-22T12:10:04Z)
• Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificate [6.581362609037603]
  We build a safe reinforcement learning framework to resolve constraints required by the DRC and its corresponding shield policy. We also devise a line search method to maintain safety and reach higher returns simultaneously while leveraging the shield policy.
  arXiv  Detail & Related papers  (2022-10-14T06:16:53Z)
• Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning [14.702446153750497]
  Worst-case-aware Robust RL (WocaR-RL) is a robust training framework for deep reinforcement learning. We show that WocaR-RL achieves state-of-the-art performance under various strong attacks.
  arXiv  Detail & Related papers  (2022-10-12T05:24:46Z)
• Policy Smoothing for Provably Robust Reinforcement Learning [109.90239627115336]
  We study the provable robustness of reinforcement learning against norm-bounded adversarial perturbations of the inputs. We generate certificates that guarantee that the total reward obtained by the smoothed policy will not fall below a certain threshold under a norm-bounded adversarial of perturbation the input.
  arXiv  Detail & Related papers  (2021-06-21T21:42:08Z)
• Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning [56.17667147101263]
  In real-world tasks, reinforcement learning agents encounter situations that are not present during training time. To ensure reliable performance, the RL agents need to exhibit robustness against worst-case situations. We propose the Robust Hallucinated Upper-Confidence RL (RH-UCRL) algorithm to provably solve this problem.
  arXiv  Detail & Related papers  (2021-03-18T16:50:17Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.