Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel

• URL: http://arxiv.org/abs/2409.18000v1
• Date: Thu, 26 Sep 2024 16:09:19 GMT
• Title: Safe Time-Varying Optimization based on Gaussian Processes with Spatio-Temporal Kernel
• Authors: Jialin Li, Marta Zagorowska, Giulia De Pasquale, Alisa Rupenyan, John Lygeros,
• Abstract summary: TVSafeOpt is an algorithm for time-varying optimization problems with unknown reward and safety functions. TVSafeOpt is capable of safely tracking a time-varying safe region without need for explicit change detection. We show that TVSafeOpt compares favorably against SafeOpt on synthetic data, both regarding safety and optimality.
• Score: 4.586346034304039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensuring safety is a key aspect in sequential decision making problems, such as robotics or process control. The complexity of the underlying systems often makes finding the optimal decision challenging, especially when the safety-critical system is time-varying. Overcoming the problem of optimizing an unknown time-varying reward subject to unknown time-varying safety constraints, we propose TVSafeOpt, a new algorithm built on Bayesian optimization with a spatio-temporal kernel. The algorithm is capable of safely tracking a time-varying safe region without the need for explicit change detection. Optimality guarantees are also provided for the algorithm when the optimization problem becomes stationary. We show that TVSafeOpt compares favorably against SafeOpt on synthetic data, both regarding safety and optimality. Evaluation on a realistic case study with gas compressors confirms that TVSafeOpt ensures safety when solving time-varying optimization problems with unknown reward and safety functions.

Related papers

• CMA-ES for Safe Optimization [9.130749109828717]
  This study focuses on CMA-ES as an efficient evolutionary algorithm and proposes an optimization method termed safe CMA-ES. The safe CMA-ES is designed to achieve both safety and efficiency in safe optimization.
  arXiv  Detail & Related papers  (2024-05-17T04:24:56Z)
• On Safety in Safe Bayesian Optimization [5.9045432488022485]
  We investigate three safety-related issues of the popular class of SafeOpt-type algorithms. First, these algorithms critically rely on frequentist bounds uncertainty for Gaussian Process (GP) regression. Second, we identify assuming an upper bound on the reproducing kernel Hilbert space (RKHS) norm of the target function. Third, SafeOpt and derived algorithms rely on a discrete search space, making them difficult to apply to higher-dimensional problems.
  arXiv  Detail & Related papers  (2024-03-19T17:50:32Z)
• Information-Theoretic Safe Bayesian Optimization [59.758009422067005]
  We consider a sequential decision making task, where the goal is to optimize an unknown function without evaluating parameters that violate an unknown (safety) constraint. Most current methods rely on a discretization of the domain and cannot be directly extended to the continuous case. We propose an information-theoretic safe exploration criterion that directly exploits the GP posterior to identify the most informative safe parameters to evaluate.
  arXiv  Detail & Related papers  (2024-02-23T14:31:10Z)
• Bayesian Optimization with Formal Safety Guarantees via Online Conformal Prediction [36.14499894307206]
  Black-box zero-th order optimization is a central primitive for applications in fields as diverse as finance, physics, and engineering. In this paper, we study scenarios in which feedback is also provided on the safety of the attempted solution. A novel BO-based approach is introduced that satisfies safety requirements irrespective of properties of the constraint function.
  arXiv  Detail & Related papers  (2023-06-30T17:26:49Z)
• Adaptive Real Time Exploration and Optimization for Safety-Critical Systems [0.0]
  We propose the ARTEO algorithm, where we cast multi-armed bandits as a programming problem subject to safety constraints. We learn the environmental characteristics through changes in optimization inputs and through exploration. Compared to existing safe-learning approaches, our algorithm does not require an exclusive exploration phase and follows the optimization goals even in the explored points.
  arXiv  Detail & Related papers  (2022-11-10T11:37:22Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning [72.97229770329214]
  We introduce a general approach for seeking high dimensional non-linear optimization problems in which maintaining safety during learning is crucial. Our approach called LBSGD is based on applying a logarithmic barrier approximation with a carefully chosen step size. We demonstrate the effectiveness of our approach on minimizing violation in policy tasks in safe reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-21T11:14:47Z)
• Towards Safe Policy Improvement for Non-Stationary MDPs [48.9966576179679]
  Many real-world problems of interest exhibit non-stationarity, and when stakes are high, the cost associated with a false stationarity assumption may be unacceptable. We take the first steps towards ensuring safety, with high confidence, for smoothly-varying non-stationary decision problems. Our proposed method extends a type of safe algorithm, called a Seldonian algorithm, through a synthesis of model-free reinforcement learning with time-series analysis.
  arXiv  Detail & Related papers  (2020-10-23T20:13:51Z)
• Robust Reinforcement Learning with Wasserstein Constraint [49.86490922809473]
  We show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm. The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.
  arXiv  Detail & Related papers  (2020-06-01T13:48:59Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.