Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deployment

• URL: http://arxiv.org/abs/2605.27659v1
• Date: Tue, 26 May 2026 20:22:44 GMT
• Title: Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deployment
• Abstract summary: Deep reinforcement learning (RL) agents for many cyber-physical systems are first trained in simulators.<n>When deployed in real world environments, they often suffer from performance degradation or safety violations because of the inevitable Sim2Real gap.<n>We propose a novel reinforcement learning framework that enables safe and efficient policy transfer via probabilistic latent embeddings and dynamic policy adaptation.
• Score: 5.938653166789926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Due to limited resources and public safety concerns, deep reinforcement learning (RL) agents for many cyber-physical systems (e.g., autonomous vehicles) are first trained in simulators. However, when deployed in real world environments, they often suffer from performance degradation or safety violations because of the inevitable Sim2Real gap. Existing zero-shot approaches, such as robust safe RL and domain randomization, mitigate this issue but typically at the cost of degraded performance or residual safety risks when experiencing unmodeled system dynamics. To address these limitations, we propose a novel reinforcement learning framework that enables safe and efficient policy transfer via probabilistic latent embeddings and dynamic policy adaptation. We consider a family of Constrained Markov Decision Processes (CMDPs) under different environment contexts. By leveraging latent context variable in meta-RL, the proposed framework infers the latent representation of the environment from simulated experiences. Furthermore, it incorporates a distributional RL formulation, which allows risk levels of the deployed policy to be adjusted dynamically, based on the estimation accuracy of the latent context variable. This strategy promotes safety at the early deployment stage and improves efficiency through fast policy adaptation under the Sim2Real gap.

Related papers

• From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving [56.30087557121323]
  AlignADV is a learnability-guided closed-loop adversarial training framework.<n>It converts adversarial scenarios into resolvable and capability-aligned curricula.<n> Experiments show it reduces training steps by up to 40.6 percent.
  arXiv  Detail & Related papers  (2026-06-12T02:13:55Z)
• Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation [54.29523408543184]
  Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment.<n>Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization.<n>We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness.
  arXiv  Detail & Related papers  (2026-03-10T00:51:47Z)
• Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models [57.006252510102506]
  Reinforcement learning (RL) is a powerful framework for optimal decision-making and control but often lacks provable guarantees for safety-critical applications.<n>We introduce a novel recovery-based shielding framework that enables safe RL with a provable safety lower bound for unknown and non-linear continuous dynamical systems.
  arXiv  Detail & Related papers  (2026-02-12T22:03:35Z)
• SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer [60.19411648245077]
  We propose SPiDR, short for Sim-to-real via Pessimistic Domain Randomization.<n> SPiDR is a scalable algorithm with provable guarantees for safe sim-to-real transfer.<n>We demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.
  arXiv  Detail & Related papers  (2025-09-23T05:03:00Z)
• Safe Domain Randomization via Uncertainty-Aware Out-of-Distribution Detection and Policy Adaptation [26.241756408576684]
  Uncertainty-Aware RL (UARL) is a novel framework that prioritizes safety during training by addressing Out-Of-Distribution (OOD) detection and policy adaptation without requiring direct interactions in target domain.<n>We evaluate UARL on MuJoCo benchmarks and a quadrupedal robot, demonstrating its effectiveness in reliable OOD detection, improved performance, and enhanced sample efficiency compared to baselines.
  arXiv  Detail & Related papers  (2025-07-08T15:51:57Z)
• Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Robotics [3.7491742648742568]
  Domain randomization is a technique to facilitate the transfer of policies from simulation to real-world robotic applications.<n>We propose a method to enable safe deployment-time policy adaptation in real-world robot control.
  arXiv  Detail & Related papers  (2025-03-13T23:28:11Z)
• Mind the Gap: Towards Generalizable Autonomous Penetration Testing via Domain Randomization and Meta-Reinforcement Learning [15.619925926862235]
  GAP is a generalizable autonomous pentesting framework.<n>It aims to realizes efficient policy training in realistic environments.<n>It also trains agents capable of drawing inferences about other cases from one instance.
  arXiv  Detail & Related papers  (2024-12-05T11:24:27Z)
• Concurrent Learning of Policy and Unknown Safety Constraints in Reinforcement Learning [4.14360329494344]
  Reinforcement learning (RL) has revolutionized decision-making across a wide range of domains over the past few decades. Yet, deploying RL policies in real-world scenarios presents the crucial challenge of ensuring safety. Traditional safe RL approaches have predominantly focused on incorporating predefined safety constraints into the policy learning process. We propose a novel approach that concurrently learns a safe RL control policy and identifies the unknown safety constraint parameters of a given environment.
  arXiv  Detail & Related papers  (2024-02-24T20:01:15Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Lyapunov-based uncertainty-aware safe reinforcement learning [0.0]
  InReinforcement learning (RL) has shown a promising performance in learning optimal policies for a variety of sequential decision-making tasks. In many real-world RL problems, besides optimizing the main objectives, the agent is expected to satisfy a certain level of safety. We propose a Lyapunov-based uncertainty-aware safe RL model to address these limitations.
  arXiv  Detail & Related papers  (2021-07-29T13:08:15Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.