Learning to Satisfy Unknown Constraints in Iterative MPC

• URL: http://arxiv.org/abs/2006.05054v3
• Date: Sat, 10 Jun 2023 19:39:31 GMT
• Title: Learning to Satisfy Unknown Constraints in Iterative MPC
• Authors: Monimoy Bujarbaruah, Charlott Vallon, Francesco Borrelli
• Abstract summary: We propose a control design method for linear time-invariant systems that iteratively learns to satisfy unknown polyhedral state constraints. At each iteration of a repetitive task, the method constructs an estimate of the unknown environment constraints using collected closed-loop trajectory data. An MPC controller is then designed to robustly satisfy the estimated constraint set.
• Score: 3.306595429364865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a control design method for linear time-invariant systems that iteratively learns to satisfy unknown polyhedral state constraints. At each iteration of a repetitive task, the method constructs an estimate of the unknown environment constraints using collected closed-loop trajectory data. This estimated constraint set is improved iteratively upon collection of additional data. An MPC controller is then designed to robustly satisfy the estimated constraint set. This paper presents the details of the proposed approach, and provides robust and probabilistic guarantees of constraint satisfaction as a function of the number of executed task iterations. We demonstrate the safety of the proposed framework and explore the safety vs. performance trade-off in a detailed numerical example.

Related papers

• Online Constraint Tightening in Stochastic Model Predictive Control: A Regression Approach [49.056933332667114]
  No analytical solutions exist for chance-constrained optimal control problems. We propose a data-driven approach for learning the constraint-tightening parameters online during control. Our approach yields constraint-tightening parameters that tightly satisfy the chance constraints.
  arXiv  Detail & Related papers  (2023-10-04T16:22:02Z)
• Dual Formulation for Chance Constrained Stochastic Shortest Path with Application to Autonomous Vehicle Behavior Planning [3.655021726150368]
  The Constrained Shortest Path problem (C-SSP) is a formalism for planning in environments under certain types of operating constraints. This work's first contribution is an exact integer linear formulation for Chance-constrained policies. Third, we show that the CC-SSP formalism can be generalized to account for constraints that span through multiple time steps.
  arXiv  Detail & Related papers  (2023-02-25T16:40:00Z)
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
  This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent. Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
  arXiv  Detail & Related papers  (2023-02-15T04:14:20Z)
• Log Barriers for Safe Black-box Optimization with Application to Safe Reinforcement Learning [72.97229770329214]
  We introduce a general approach for seeking high dimensional non-linear optimization problems in which maintaining safety during learning is crucial. Our approach called LBSGD is based on applying a logarithmic barrier approximation with a carefully chosen step size. We demonstrate the effectiveness of our approach on minimizing violation in policy tasks in safe reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-21T11:14:47Z)
• Reinforcement Learning for Task Specifications with Action-Constraints [4.046919218061427]
  We propose a method to learn optimal control policies for a finite-state Markov Decision Process. We assume that the set of action sequences that are deemed unsafe and/or safe are given in terms of a finite-state automaton. We present a version of the Q-learning algorithm for learning optimal policies in the presence of non-Markovian action-sequence and state constraints.
  arXiv  Detail & Related papers  (2022-01-02T04:22:01Z)
• Controllable Summarization with Constrained Markov Decision Process [50.04321779376415]
  We study controllable text summarization which allows users to gain control on a particular attribute. We propose a novel training framework based on Constrained Markov Decision Process (CMDP) Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness.
  arXiv  Detail & Related papers  (2021-08-07T09:12:53Z)
• Shortest-Path Constrained Reinforcement Learning for Sparse Reward Tasks [59.419152768018506]
  We show that any optimal policy necessarily satisfies the k-SP constraint. We propose a novel cost function that penalizes the policy violating SP constraint, instead of completely excluding it. Our experiments on MiniGrid, DeepMind Lab, Atari, and Fetch show that the proposed method significantly improves proximal policy optimization (PPO)
  arXiv  Detail & Related papers  (2021-07-13T21:39:21Z)
• Online Learning Based Risk-Averse Stochastic MPC of Constrained Linear Uncertain Systems [0.0]
  This paper investigates the problem of designing data-driven Model Predictive Control (MPC) for linear time-invariant systems under additive disturbance. We propose a novel online learning based risk-varying MPC framework in which Conditional Value-at-Risk (CVaR) constraints are required to hold for a family of distributions called an ambiguity set.
  arXiv  Detail & Related papers  (2020-11-20T13:00:28Z)
• Constrained Model-Free Reinforcement Learning for Process Optimization [0.0]
  Reinforcement learning (RL) is a control approach that can handle nonlinear optimal control problems. Despite the promise exhibited, RL has yet to see marked translation to industrial practice. We propose an 'oracle'-assisted constrained Q-learning algorithm that guarantees the satisfaction of joint chance constraints with a high probability.
  arXiv  Detail & Related papers  (2020-11-16T13:16:22Z)
• Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPs [13.922754427601491]
  We characterize the relationship between safety constraints and the number of samples needed to ensure a desired level of accuracy. Our main finding is that compared to the best known bounds of the unconstrained regime, the sample of constrained RL algorithms are increased by a factor that is logarithmic in the number of constraints.
  arXiv  Detail & Related papers  (2020-08-01T18:17:08Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.