Safe Meta-Reinforcement Learning via Information Space Reachability

• URL: http://arxiv.org/abs/2609.15915v1
• Date: Mon, 14 Sep 2026 17:27:11 GMT
• Title: Safe Meta-Reinforcement Learning via Information Space Reachability
• Abstract summary: We propose a safe meta-RL framework that explicitly accounts for safety during adaptation.<n>We show that this function satisfies a self-consistency condition and a Bellman equation, which make it learnable via meta-RL.
• Score: 4.249024052507976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-reinforcement learning (meta-RL) enables agents to adapt to unseen tasks with limited experience. Despite its promise, the application of meta-RL in real-world tasks is hindered by safety requirements, which have been underexplored in prior work. In this paper, we propose a safe meta-RL framework that explicitly accounts for safety during adaptation. Our key insight is to reason about safety in the information space, which captures both the physical state and the agent's belief over the underlying task. Within this space, we introduce a safety value function that measures the probability of the agent avoiding unsafe regions indefinitely. We show that this function satisfies a self-consistency condition and a Bellman equation, which make it learnable via meta-RL. Based on this formulation, we develop a safe meta-RL algorithm that learns the safety value function and leverages it for safety filtering and constrained policy optimization. Experiments on meta-RL benchmarks demonstrate the effectiveness of the proposed method.

Related papers

• A Provable Approach for End-to-End Safe Reinforcement Learning [17.17447653795906]
  A longstanding goal in safe reinforcement learning (RL) is to ensure the safety of a policy throughout the entire process.<n>We propose a method, called Provably Lifetime Safe RL (PLS), that integrates offline safe RL with safe policy deployment.
  arXiv  Detail & Related papers  (2025-05-28T00:48:20Z)
• Probabilistic Shielding for Safe Reinforcement Learning [51.35559820893218]
  In real-life scenarios, a Reinforcement Learning (RL) agent must often also behave in a safe manner, including at training time.<n>We present a new, scalable method, which enjoys strict formal guarantees for Safe RL.<n>We show that our approach provides a strict formal safety guarantee that the agent stays safe at training and test time.
  arXiv  Detail & Related papers  (2025-03-09T17:54:33Z)
• ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning [48.536695794883826]
  We present ActSafe, a novel model-based RL algorithm for safe and efficient exploration.<n>We show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time.<n>In addition, we propose a practical variant of ActSafe that builds on latest model-based RL advancements.
  arXiv  Detail & Related papers  (2024-10-12T10:46:02Z)
• Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning [57.84059344739159]
  "Shielding" is a popular technique to enforce safety inReinforcement Learning (RL) We propose a new permissibility-based framework to deal with safety and shield construction.
  arXiv  Detail & Related papers  (2024-05-29T18:00:21Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Provable Safe Reinforcement Learning with Binary Feedback [62.257383728544006]
  We consider the problem of provable safe RL when given access to an offline oracle providing binary feedback on the safety of state, action pairs. We provide a novel meta algorithm, SABRE, which can be applied to any MDP setting given access to a blackbox PAC RL algorithm for that setting.
  arXiv  Detail & Related papers  (2022-10-26T05:37:51Z)
• Safe Reinforcement Learning via Confidence-Based Filters [78.39359694273575]
  We develop a control-theoretic approach for certifying state safety constraints for nominal policies learned via standard reinforcement learning techniques. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-07-04T11:43:23Z)
• Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking [12.719948223824483]
  reinforcement learning (RL) algorithms are crucial to unlock their potential for many real-world tasks. However, vanilla RL and most safe RL approaches do not guarantee safety. We introduce a categorization of existing provably safe RL methods, present the conceptual foundations for both continuous and discrete action spaces, and empirically benchmark existing methods. We provide practical guidance on selecting provably safe RL approaches depending on the safety specification, RL algorithm, and type of action space.
  arXiv  Detail & Related papers  (2022-05-13T16:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.