Safe Control and Learning Using Generalized Action Governor

• URL: http://arxiv.org/abs/2211.12628v1
• Date: Tue, 22 Nov 2022 23:25:25 GMT
• Title: Safe Control and Learning Using Generalized Action Governor
• Abstract summary: This paper introduces the Generalized Action Governor, which is a supervisory scheme for augmenting a nominal closed-loop system with the capability of strictly handling constraints. We discuss its application to safe online learning, which aims to safely evolve control parameters using real-time data to improve performance for uncertain systems.
• Score: 5.7406123763046955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces the Generalized Action Governor, which is a supervisory scheme for augmenting a nominal closed-loop system with the capability of strictly handling constraints. After presenting its theory for general systems and introducing tailored design approaches for linear and discrete systems, we discuss its application to safe online learning, which aims to safely evolve control parameters using real-time data to improve performance for uncertain systems. In particular, we propose two safe learning algorithms based on integration of reinforcement learning/data-driven Koopman operator-based control with the generalized action governor. The developments are illustrated with a numerical example.

Related papers

• SoK: Harmonizing Attack Graphs and Intrusion Detection Systems [70.34940998791319]
  Attack Graphs (AGs) serve as the primary threat model for analyzing attacker strategies and informing any response.<n>Despite the conceptual connection being recognized in early research, the field of AG and IDS integration lacks a common structure.<n>This paper presents the first systematic analysis of AG-IDS integration, reviewing a 73 comprehensive works in literature.
  arXiv  Detail & Related papers  (2026-03-09T12:16:51Z)
• SAGE-LLM: Towards Safe and Generalizable LLM Controller with Fuzzy-CBF Verification and Graph-Structured Knowledge Retrieval for UAV Decision [46.089736018739295]
  Large Language Models (LLM) lack domain-specific UAV control knowledge and formal safety assurances.<n>This paper proposes a train-free two-layer decision architecture based on LLMs, integrating high-level safety planning with low-level precise control.
  arXiv  Detail & Related papers  (2026-02-27T06:41:04Z)
• Toward General Instruction-Following Alignment for Retrieval-Augmented Generation [63.611024451010316]
  Following natural instructions is crucial for the effective application of Retrieval-Augmented Generation (RAG) systems. We propose VIF-RAG, the first automated, scalable, and verifiable synthetic pipeline for instruction-following alignment in RAG systems.
  arXiv  Detail & Related papers  (2024-10-12T16:30:51Z)
• Last-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning [62.81324245896717]
  We introduce an exploration-agnostic algorithm, called C-PG, which exhibits global last-ite convergence guarantees under (weak) gradient domination assumptions. We numerically validate our algorithms on constrained control problems, and compare them with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-07-15T14:54:57Z)
• Nonparametric Control-Koopman Operator Learning: Flexible and Scalable Models for Prediction and Control [2.7784144651669704]
  We present a nonparametric framework for learning Koopman operator representations of nonlinear control-affine systems. We also enhance the scalability of control-Koopman operator estimators by leveraging random projections. The efficacy of our novel cKOR approach is demonstrated on both forecasting and control tasks.
  arXiv  Detail & Related papers  (2024-05-12T15:46:52Z)
• Data-Driven Permissible Safe Control with Barrier Certificates [11.96747040086603]
  This paper introduces a method of identifying a maximal set of safe strategies from data for systems with unknown dynamics. Case studies show that increasing the size of the dataset for learning the system grows the permissible strategy set.
  arXiv  Detail & Related papers  (2024-04-30T18:32:24Z)
• A Tutorial on Gaussian Process Learning-based Model Predictive Control [6.868436894461309]
  This tutorial introduces the advanced approach integrating Gaussian process (GP) with model predictive control (MPC) for enhanced control in complex systems. A central contribution of this tutorial is the first detailed, systematic mathematical formulation of GP-MPC in literature. Practical applications in robotics control, such as path-following for mobile robots in challenging terrains and mixed-vehicle platooning, are discussed.
  arXiv  Detail & Related papers  (2024-04-02T03:13:05Z)
• Decentralized Event-Triggered Online Learning for Safe Consensus of Multi-Agent Systems with Gaussian Process Regression [3.405252606286664]
  This paper presents a novel learning-based distributed control law, augmented by an auxiliary dynamics. For continuous enhancement in predictive performance, a data-efficient online learning strategy with a decentralized event-triggered mechanism is proposed. To demonstrate the efficacy of the proposed learning-based controller, a comparative analysis is conducted, contrasting it with both conventional distributed control laws and offline learning methodologies.
  arXiv  Detail & Related papers  (2024-02-05T16:41:17Z)
• Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory [46.85103495283037]
  We propose a new approach to apply verification methods from control theory to learned value functions. We formalize original theorems that establish links between value functions and control barrier functions. Our work marks a significant step towards a formal framework for the general, scalable, and verifiable design of RL-based control systems.
  arXiv  Detail & Related papers  (2023-06-06T21:41:31Z)
• In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States [84.24300005271185]
  We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
  arXiv  Detail & Related papers  (2023-01-27T22:28:19Z)
• Robust Action Governor for Uncertain Piecewise Affine Systems with Non-convex Constraints and Safe Reinforcement Learning [5.7406123763046955]
  The action governor is an add-on scheme to a nominal control loop that monitors and adjusts the control actions to enforce safety specifications expressed as point-in-time state and constraints. We introduce the Robust Action Governor (RAG) of systems which can be represented as non-time additive uncertainties. This development enables safe real-time evolution of the environment and adaptation to changes in system parameters.
  arXiv  Detail & Related papers  (2022-07-17T17:31:07Z)
• Reinforcement Learning for Task Specifications with Action-Constraints [4.046919218061427]
  We propose a method to learn optimal control policies for a finite-state Markov Decision Process. We assume that the set of action sequences that are deemed unsafe and/or safe are given in terms of a finite-state automaton. We present a version of the Q-learning algorithm for learning optimal policies in the presence of non-Markovian action-sequence and state constraints.
  arXiv  Detail & Related papers  (2022-01-02T04:22:01Z)
• Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations [50.37808220291108]
  This paper addresses learning safe output feedback control laws from partial observations of expert demonstrations. We first propose robust output control barrier functions (ROCBFs) as a means to guarantee safety. We then formulate an optimization problem to learn ROCBFs from expert demonstrations that exhibit safe system behavior.
  arXiv  Detail & Related papers  (2021-11-18T23:21:00Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Adaptive control of a mechatronic system using constrained residual reinforcement learning [0.0]
  We propose a simple, practical and intuitive approach to improve the performance of a conventional controller in uncertain environments. Our approach is motivated by the observation that conventional controllers in industrial motion control value robustness over adaptivity to deal with different operating conditions.
  arXiv  Detail & Related papers  (2021-10-06T08:13:05Z)
• Safe RAN control: A Symbolic Reinforcement Learning Approach [62.997667081978825]
  We present a Symbolic Reinforcement Learning (SRL) based architecture for safety control of Radio Access Network (RAN) applications. We provide a purely automated procedure in which a user can specify high-level logical safety specifications for a given cellular network topology. We introduce a user interface (UI) developed to help a user set intent specifications to the system, and inspect the difference in agent proposed actions.
  arXiv  Detail & Related papers  (2021-06-03T16:45:40Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)
• Learning Hybrid Control Barrier Functions from Data [66.37785052099423]
  Motivated by the lack of systematic tools to obtain safe control laws for hybrid systems, we propose an optimization-based framework for learning certifiably safe control laws from data. In particular, we assume a setting in which the system dynamics are known and in which data exhibiting safe system behavior is available.
  arXiv  Detail & Related papers  (2020-11-08T23:55:02Z)
• Control Barrier Functions for Unknown Nonlinear Systems using Gaussian Processes [17.870440210358847]
  This paper focuses on the controller synthesis for unknown, nonlinear systems while ensuring safety constraints. In the learning step, we use a data-driven approach to learn the unknown control affine nonlinear dynamics together with a statistical bound on the accuracy of the learned model. In the second controller synthesis steps, we develop a systematic approach to compute control barrier functions that explicitly take into consideration the uncertainty of the learned model.
  arXiv  Detail & Related papers  (2020-10-12T16:12:52Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.