Local Bayesian Optimization for Controller Tuning with Crash Constraints

• URL: http://arxiv.org/abs/2411.16267v1
• Date: Mon, 25 Nov 2024 10:37:48 GMT
• Title: Local Bayesian Optimization for Controller Tuning with Crash Constraints
• Authors: Alexander von Rohr, David Stenger, Dominik Scheurenberg, Sebastian Trimpe,
• Abstract summary: We extend a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region. Our findings showcase the potential of local BO to enhance controller performance and reduce the time and resources necessary for tuning.
• Score: 47.30677525394649
• License:
• Abstract: Controller tuning is crucial for closed-loop performance but often involves manual adjustments. Although Bayesian optimization (BO) has been established as a data-efficient method for automated tuning, applying it to large and high-dimensional search spaces remains challenging. We extend a recently proposed local variant of BO to include crash constraints, where the controller can only be successfully evaluated in an a-priori unknown feasible region. We demonstrate the efficiency of the proposed method through simulations and hardware experiments. Our findings showcase the potential of local BO to enhance controller performance and reduce the time and resources necessary for tuning.

Related papers

• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• Violation-Aware Contextual Bayesian Optimization for Controller Performance Optimization with Unmodeled Constraints [1.8730951928453339]
  We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. We propose a violation-aware contextual BO algorithm (VACBO) that optimize closed-loop performance while simultaneously learning constraint-feasible solutions. We demonstrate the effectiveness of our proposed VACBO method for energy minimization of industrial vapor compression systems under time-varying ambient temperature and humidity.
  arXiv  Detail & Related papers  (2023-01-28T05:48:40Z)
• Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning [6.5158195776494]
  We tackle the controller tuning problem using a novel derivative-free reinforcement learning framework. We conduct numerical experiments on two concrete examples from autonomous driving, namely, adaptive cruise control with PID controller and trajectory tracking with MPC controller. Experimental results show that the proposed method outperforms popular baselines and highlight its strong potential for controller tuning.
  arXiv  Detail & Related papers  (2022-09-11T13:01:14Z)
• On Controller Tuning with Time-Varying Bayesian Optimization [74.57758188038375]
  We will use time-varying optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. We propose a novel TVBO strategy using Uncertainty-Injection (UI), which incorporates the assumption of incremental and lasting changes. Our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and fewer unstable parameter configurations.
  arXiv  Detail & Related papers  (2022-07-22T14:54:13Z)
• Tuning Particle Accelerators with Safety Constraints using Bayesian Optimization [73.94660141019764]
  tuning machine parameters of particle accelerators is a repetitive and time-consuming task. We propose and evaluate a step size-limited variant of safe Bayesian optimization.
  arXiv  Detail & Related papers  (2022-03-26T02:21:03Z)
• VABO: Violation-Aware Bayesian Optimization for Closed-Loop Control Performance Optimization with Unmodeled Constraints [2.205124036454768]
  We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. We propose a violation-aware BO algorithm (VABO) that optimize closed-loop performance while simultaneously learning constraint-feasible solutions. We demonstrate the effectiveness of our proposed VABO method for energy minimization of industrial vapor compression systems.
  arXiv  Detail & Related papers  (2021-10-14T15:51:03Z)
• Regret-optimal Estimation and Control [52.28457815067461]
  We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form. We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics.
  arXiv  Detail & Related papers  (2021-06-22T23:14:21Z)
• Safe and Efficient Model-free Adaptive Control via Bayesian Optimization [39.962395119933596]
  We propose a purely data-driven, model-free approach for adaptive control. tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation.
  arXiv  Detail & Related papers  (2021-01-19T19:15:00Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.