Robust nonlinear set-point control with reinforcement learning

• URL: http://arxiv.org/abs/2304.10277v1
• Date: Thu, 20 Apr 2023 13:00:04 GMT
• Title: Robust nonlinear set-point control with reinforcement learning
• Authors: Ruoqi Zhang, Per Mattsson, Torbj\"orn Wigren
• Abstract summary: This paper argues that three ideas can improve reinforcement learning methods even for highly nonlinear set-point control problems. The claim is supported by experiments with a real-world nonlinear cascaded tank process and a simulated strongly nonlinear pH-control system.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There has recently been an increased interest in reinforcement learning for nonlinear control problems. However standard reinforcement learning algorithms can often struggle even on seemingly simple set-point control problems. This paper argues that three ideas can improve reinforcement learning methods even for highly nonlinear set-point control problems: 1) Make use of a prior feedback controller to aid amplitude exploration. 2) Use integrated errors. 3) Train on model ensembles. Together these ideas lead to more efficient training, and a trained set-point controller that is more robust to modelling errors and thus can be directly deployed to real-world nonlinear systems. The claim is supported by experiments with a real-world nonlinear cascaded tank process and a simulated strongly nonlinear pH-control system.

Related papers

• Sample-efficient diffusion-based control of complex nonlinear systems [12.75120974078924]
  SEDC is a novel diffusion-based control framework addressing high-dimensional state-action spaces, nonlinear system dynamics, and the gap between non-optimal training data and near-optimal control solutions. Our approach represents a significant advancement in sample-efficient control of complex nonlinear systems.
  arXiv  Detail & Related papers  (2025-02-25T06:30:04Z)
• Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States [52.56827348431552]
  gradient descent frequently exhibits an implicit bias that leads to excellent performance on unseen data. This paper theoretically studies the implicit bias of policy gradient in terms of extrapolation to unseen initial states.
  arXiv  Detail & Related papers  (2024-02-12T18:41:31Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• Aiding reinforcement learning for set point control [0.0]
  The paper contributes by augmentation of reinforcement learning with a simple guiding feedback controller. The proposed method is evaluated with simulation and on a real-world double tank process with promising results.
  arXiv  Detail & Related papers  (2023-04-20T13:12:00Z)
• A stabilizing reinforcement learning approach for sampled systems with partially unknown models [0.0]
  We suggest a method to guarantee practical stability of the system-controller closed loop in a purely online learning setting. To achieve the claimed results, we employ techniques of classical adaptive control. The method is tested in adaptive traction control and cruise control where it proved to significantly reduce the cost.
  arXiv  Detail & Related papers  (2022-08-31T09:20:14Z)
• Improving the Performance of Robust Control through Event-Triggered Learning [74.57758188038375]
  We propose an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem. We demonstrate improved performance over a robust controller baseline in a numerical example.
  arXiv  Detail & Related papers  (2022-07-28T17:36:37Z)
• Comparative analysis of machine learning methods for active flow control [60.53767050487434]
  Genetic Programming (GP) and Reinforcement Learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, bench-marking some of their most representative algorithms against global optimization techniques.
  arXiv  Detail & Related papers  (2022-02-23T18:11:19Z)
• Deep Koopman Operator with Control for Nonlinear Systems [44.472875714432504]
  We propose an end-to-end deep learning framework to learn the Koopman embedding function and Koopman Operator. We first parameterize the embedding function and Koopman Operator with the neural network and train them end-to-end with the K-steps loss function. We then design an auxiliary control network to encode the nonlinear state-dependent control term to model the nonlinearity in control input.
  arXiv  Detail & Related papers  (2022-02-16T11:40:36Z)
• Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments [0.37798600249187286]
  We show that DRL can successfully learn to perform attitude control of a fixed-wing UAV operating directly on the original nonlinear dynamics. We deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller.
  arXiv  Detail & Related papers  (2021-11-07T19:07:46Z)
• Reinforcement Learning for Control of Valves [0.0]
  This paper is a study of reinforcement learning (RL) as an optimal-control strategy for control of nonlinear valves. It is evaluated against the PID (proportional-integral-derivative) strategy, using a unified framework.
  arXiv  Detail & Related papers  (2020-12-29T09:01:47Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.