Reinforcement Learning based Design of Linear Fixed Structure Controllers

• URL: http://arxiv.org/abs/2005.04537v1
• Date: Sun, 10 May 2020 00:53:11 GMT
• Title: Reinforcement Learning based Design of Linear Fixed Structure Controllers
• Authors: Nathan P. Lawrence, Gregory E. Stewart, Philip D. Loewen, Michael G. Forbes, Johan U. Backstrom, R. Bhushan Gopaluni
• Abstract summary: We present a simple finite-difference approach, based on random search, to tuning linear fixed-structure controllers. Our algorithm operates on the entire closed-loop step response of the system and iteratively improves the PID gains towards a desired closed-loop response.
• Score: 3.131740922192114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning has been successfully applied to the problem of tuning PID controllers in several applications. The existing methods often utilize function approximation, such as neural networks, to update the controller parameters at each time-step of the underlying process. In this work, we present a simple finite-difference approach, based on random search, to tuning linear fixed-structure controllers. For clarity and simplicity, we focus on PID controllers. Our algorithm operates on the entire closed-loop step response of the system and iteratively improves the PID gains towards a desired closed-loop response. This allows for embedding stability requirements into the reward function without any modeling procedures.

Related papers

• Continual Model-based Reinforcement Learning for Data Efficient Wireless Network Optimisation [73.04087903322237]
  We formulate throughput optimisation as Continual Reinforcement Learning of control policies. Simulation results suggest that the proposed system is able to shorten the end-to-end deployment lead-time by two-fold.
  arXiv  Detail & Related papers  (2024-04-30T11:23:31Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• ReACT: Reinforcement Learning for Controller Parametrization using B-Spline Geometries [0.0]
  This work presents a novel approach using deep reinforcement learning (DRL) with N-dimensional B-spline geometries (BSGs) We focus on the control of parameter-variant systems, a class of systems with complex behavior which depends on the operating conditions. We make the adaptation process more efficient by introducing BSGs to map the controller parameters which may depend on numerous operating conditions.
  arXiv  Detail & Related papers  (2024-01-10T16:27:30Z)
• Self-Tuning PID Control via a Hybrid Actor-Critic-Based Neural Structure for Quadcopter Control [0.0]
  Proportional-Integrator-Derivative (PID) controller is used in a wide range of industrial and experimental processes. Due to the uncertainty of model parameters and external disturbances, real systems such as Quadrotors need more robust and reliable PID controllers. In this research, a self-tuning PID controller using a Reinforcement-Learning-based Neural Network has been investigated.
  arXiv  Detail & Related papers  (2023-07-03T19:35:52Z)
• Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning [6.5158195776494]
  We tackle the controller tuning problem using a novel derivative-free reinforcement learning framework. We conduct numerical experiments on two concrete examples from autonomous driving, namely, adaptive cruise control with PID controller and trajectory tracking with MPC controller. Experimental results show that the proposed method outperforms popular baselines and highlight its strong potential for controller tuning.
  arXiv  Detail & Related papers  (2022-09-11T13:01:14Z)
• DiffLoop: Tuning PID controllers by differentiating through the feedback loop [8.477619837043214]
  This paper investigates PID tuning and anti-windup measures. In particular, we use a cost function and generate gradients to improve controller performance.
  arXiv  Detail & Related papers  (2021-06-19T15:26:46Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: from Time-Driven to Event-Driven [80.94390916562179]
  Time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives. It is desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise. We show how the event-driven dHDP algorithm works in comparison to the original time-driven dHDP.
  arXiv  Detail & Related papers  (2020-06-16T05:51:25Z)
• Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem [3.131740922192114]
  We focus on the interpretability of DRL control methods. In particular, we view linear fixed-structure controllers as shallow neural networks embedded in the actor-critic framework.
  arXiv  Detail & Related papers  (2020-05-10T01:05:26Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework [108.4560749465701]
  The Pontryagin Differentiable Programming methodology establishes a unified framework to solve a broad class of learning and control tasks. We investigate three learning modes of the PDP: inverse reinforcement learning, system identification, and control/planning. We demonstrate the capability of the PDP in each learning mode on different high-dimensional systems, including multi-link robot arm, 6-DoF maneuvering quadrotor, and 6-DoF rocket powered landing.
  arXiv  Detail & Related papers  (2019-12-30T15:35:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.