The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems

• URL: http://arxiv.org/abs/2309.05837v1
• Date: Mon, 11 Sep 2023 21:34:16 GMT
• Title: The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems
• Authors: Kai-Chieh Hsu, Haimin Hu, Jaime Fern\'andez Fisac
• Abstract summary: This article reviews safety filter approaches and proposes a unified technical framework to understand, compare, and combine them. The new unified view exposes a shared modular structure across a range of seemingly disparate safety filter classes. It naturally suggests directions for future progress towards more scalable synthesis, robust monitoring, and efficient intervention.
• Score: 0.08521820162570426
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent years have seen significant progress in the realm of robot autonomy, accompanied by the expanding reach of robotic technologies. However, the emergence of new deployment domains brings unprecedented challenges in ensuring safe operation of these systems, which remains as crucial as ever. While traditional model-based safe control methods struggle with generalizability and scalability, emerging data-driven approaches tend to lack well-understood guarantees, which can result in unpredictable catastrophic failures. Successful deployment of the next generation of autonomous robots will require integrating the strengths of both paradigms. This article provides a review of safety filter approaches, highlighting important connections between existing techniques and proposing a unified technical framework to understand, compare, and combine them. The new unified view exposes a shared modular structure across a range of seemingly disparate safety filter classes and naturally suggests directions for future progress towards more scalable synthesis, robust monitoring, and efficient intervention.

Related papers

• A New Perspective On AI Safety Through Control Theory Methodologies [16.51699616065134]
  AI promises to achieve a new level of autonomy but is hampered by a lack of safety assurance.<n>This article outlines a new perspective on AI safety based on an interdisciplinary interpretation of the underlying data-generation process.<n>The new perspective, also referred to as data control, aims to stimulate AI engineering to take advantage of existing safety analysis and assurance.
  arXiv  Detail & Related papers  (2025-06-30T10:26:59Z)
• Towards provable probabilistic safety for scalable embodied AI systems [79.31011047593492]
  Embodied AI systems are increasingly prevalent across various applications.<n> Ensuring their safety in complex operating environments remains a major challenge.<n>This Perspective offers a pathway toward safer, large-scale adoption of embodied AI systems in safety-critical applications.
  arXiv  Detail & Related papers  (2025-06-05T15:46:25Z)
• Engineering Risk-Aware, Security-by-Design Frameworks for Assurance of Large-Scale Autonomous AI Models [0.0]
  This paper presents an enterprise-level, risk-aware, security-by-design approach for large-scale autonomous AI systems.<n>We detail a unified pipeline that delivers provable guarantees of model behavior under adversarial and operational stress.<n>Case studies in national security, open-source model governance, and industrial automation demonstrate measurable reductions in vulnerability and compliance overhead.
  arXiv  Detail & Related papers  (2025-05-09T20:14:53Z)
• Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation [55.02966123945644]
  We propose a hierarchical control framework leveraging neural network verification techniques to design control barrier functions (CBFs) and policy correction mechanisms. Our approach relies on probabilistic enumeration to identify unsafe regions of operation, which are then used to construct a safe CBF-based control layer. These experiments demonstrate the ability of the proposed solution to correct unsafe actions while preserving efficient navigation behavior.
  arXiv  Detail & Related papers  (2025-04-30T13:47:25Z)
• An Approach to Technical AGI Safety and Security [72.83728459135101]
  We develop an approach to address the risk of harms consequential enough to significantly harm humanity. We focus on technical approaches to misuse and misalignment. We briefly outline how these ingredients could be combined to produce safety cases for AGI systems.
  arXiv  Detail & Related papers  (2025-04-02T15:59:31Z)
• Safe LLM-Controlled Robots with Formal Guarantees via Reachability Analysis [0.6749750044497732]
  This paper introduces a safety assurance framework for Large Language Models (LLMs)-controlled robots based on data-driven reachability analysis. Our approach provides rigorous safety guarantees against unsafe behaviors without relying on explicit analytical models.
  arXiv  Detail & Related papers  (2025-03-05T21:23:15Z)
• Learning Vision-Based Neural Network Controllers with Semi-Probabilistic Safety Guarantees [24.650302053973142]
  We introduce a novel semi-probabilistic verification framework that integrates reachability analysis with conditional generative adversarial networks. Next, we develop a gradient-based training approach that employs a novel safety loss function, safety-aware data-sampling strategy, and curriculum learning. Empirical evaluations in X-Plane 11 airplane landing simulation, CARLA-simulated autonomous lane following, and F1Tenth lane following in a visually-rich miniature environment demonstrate the effectiveness of our method in achieving formal safety guarantees.
  arXiv  Detail & Related papers  (2025-02-28T21:16:42Z)
• Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis [6.267574471145217]
  Hamilton-Jacobi (H) reachability is a rigorous framework that enables robots to simultaneously detect unsafe states and generate actions. We propose La Safety Filters, a latent-space reachability that operates directly on raw observation data.
  arXiv  Detail & Related papers  (2025-02-02T22:00:20Z)
• In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models [97.82118821263825]
  Text-to-image (T2I) models have shown remarkable progress, but their potential to generate harmful content remains a critical concern in the ML community. We propose ICER, a novel red-teaming framework that generates interpretable and semantic meaningful problematic prompts. Our work provides crucial insights for developing more robust safety mechanisms in T2I systems.
  arXiv  Detail & Related papers  (2024-11-25T04:17:24Z)
• Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning [17.856459823003277]
  Safety is one of the key issues preventing the deployment of reinforcement learning techniques in real-world robots. In this paper, we bridge the gap by extending the safe exploration method, ATACOM, with learnable constraints.
  arXiv  Detail & Related papers  (2024-09-18T15:08:41Z)
• ABNet: Attention BarrierNet for Safe and Scalable Robot Learning [58.4951884593569]
  Barrier-based method is one of the dominant approaches for safe robot learning. We propose Attention BarrierNet (ABNet) that is scalable to build larger foundational safe models in an incremental manner. We demonstrate the strength of ABNet in 2D robot obstacle avoidance, safe robot manipulation, and vision-based end-to-end autonomous driving.
  arXiv  Detail & Related papers  (2024-06-18T19:37:44Z)
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems [88.80306881112313]
  We will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them.
  arXiv  Detail & Related papers  (2024-05-10T17:38:32Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States [84.24300005271185]
  We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
  arXiv  Detail & Related papers  (2023-01-27T22:28:19Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• An Empirical Analysis of the Use of Real-Time Reachability for the Safety Assurance of Autonomous Vehicles [7.1169864450668845]
  We propose using a real-time reachability algorithm for the implementation of the simplex architecture to assure the safety of a 1/10 scale open source autonomous vehicle platform. In our approach, the need to analyze an underlying controller is abstracted away, instead focusing on the effects of the controller's decisions on the system's future states.
  arXiv  Detail & Related papers  (2022-05-03T11:12:29Z)
• Safety-aware Policy Optimisation for Autonomous Racing [17.10371721305536]
  We introduce Hamilton-Jacobi (HJ) reachability theory into the constrained Markov decision process (CMDP) framework. We demonstrate that the HJ safety value can be learned directly on vision context. We evaluate our method on several benchmark tasks, including Safety Gym and Learn-to-Race (L2R), a recently-released high-fidelity autonomous racing environment.
  arXiv  Detail & Related papers  (2021-10-14T20:15:45Z)
• Safe Active Dynamics Learning and Control: A Sequential Exploration-Exploitation Framework [30.58186749790728]
  We propose a theoretically-justified approach to maintaining safety in the presence of dynamics uncertainty. Our framework guarantees the high-probability satisfaction of all constraints at all times jointly. This theoretical analysis also motivates two regularizers of last-layer meta-learning models that improve online adaptation capabilities.
  arXiv  Detail & Related papers  (2020-08-26T17:39:58Z)
• Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approach [2.741266294612776]
  We propose a model-free safety specification method that learns the maximal probability of safe operation. Our approach constructs a Lyapunov function with respect to a safe policy to restrain each policy improvement stage. It yields a sequence of safe policies that determine the range of safe operation, called the safe set.
  arXiv  Detail & Related papers  (2020-02-24T09:20:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.