Pontryagin Optimal Control via Neural Networks
- URL: http://arxiv.org/abs/2212.14566v3
- Date: Mon, 15 Jan 2024 06:09:01 GMT
- Title: Pontryagin Optimal Control via Neural Networks
- Authors: Chengyang Gu, Hui Xiong and Yize Chen
- Abstract summary: We integrate Neural Networks with the Pontryagin's Maximum Principle (PMP), and propose a sample efficient framework NN-PMP-Gradient.
The resulting controller can be implemented for systems with unknown and complex dynamics.
Compared with the widely applied model-free and model-based reinforcement learning (RL) algorithms, our NN-PMP-Gradient achieves higher sample-efficiency and performance in terms of control objectives.
- Score: 19.546571122359534
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Solving real-world optimal control problems are challenging tasks, as the
complex, high-dimensional system dynamics are usually unrevealed to the
decision maker. It is thus hard to find the optimal control actions
numerically. To deal with such modeling and computation challenges, in this
paper, we integrate Neural Networks with the Pontryagin's Maximum Principle
(PMP), and propose a sample efficient framework NN-PMP-Gradient. The resulting
controller can be implemented for systems with unknown and complex dynamics. By
taking an iterative approach, the proposed framework not only utilizes the
accurate surrogate models parameterized by neural networks, it also efficiently
recovers the optimality conditions along with the optimal action sequences via
PMP conditions. Numerical simulations on Linear Quadratic Regulator, energy
arbitrage of grid-connected lossy battery, control of single pendulum, and two
MuJoCo locomotion tasks demonstrate our proposed NN-PMP-Gradient is a general
and versatile computation tool for finding optimal solutions. And compared with
the widely applied model-free and model-based reinforcement learning (RL)
algorithms, our NN-PMP-Gradient achieves higher sample-efficiency and
performance in terms of control objectives.
Related papers
- A neural network-based approach to hybrid systems identification for control [4.324244627273018]
We propose a specific neural network architecture that yields a hybrid system with piecewise-affine dynamics.
We show that our NN-based technique enjoys very similar performance to state-of-the-art system identification methodologies for hybrid systems.
arXiv Detail & Related papers (2024-04-02T10:16:30Z) - Model-Based Control with Sparse Neural Dynamics [23.961218902837807]
We propose a new framework for integrated model learning and predictive control.
We show that our framework can deliver better closed-loop performance than existing state-of-the-art methods.
arXiv Detail & Related papers (2023-12-20T06:25:02Z) - Pointer Networks with Q-Learning for Combinatorial Optimization [55.2480439325792]
We introduce the Pointer Q-Network (PQN), a hybrid neural architecture that integrates model-free Q-value policy approximation with Pointer Networks (Ptr-Nets)
Our empirical results demonstrate the efficacy of this approach, also testing the model in unstable environments.
arXiv Detail & Related papers (2023-11-05T12:03:58Z) - Robust optimal well control using an adaptive multi-grid reinforcement
learning framework [0.0]
Reinforcement learning is a promising tool to solve robust optimal well control problems.
The proposed framework is demonstrated using a state-of-the-art, model-free policy-based RL algorithm.
Prominent gains in the computational efficiency is observed using the proposed framework saving around 60-70% of computational cost of its single fine-grid counterpart.
arXiv Detail & Related papers (2022-07-07T12:08:57Z) - Real-time Neural-MPC: Deep Learning Model Predictive Control for
Quadrotors and Agile Robotic Platforms [59.03426963238452]
We present Real-time Neural MPC, a framework to efficiently integrate large, complex neural network architectures as dynamics models within a model-predictive control pipeline.
We show the feasibility of our framework on real-world problems by reducing the positional tracking error by up to 82% when compared to state-of-the-art MPC approaches without neural network dynamics.
arXiv Detail & Related papers (2022-03-15T09:38:15Z) - High-dimensional Bayesian Optimization Algorithm with Recurrent Neural
Network for Disease Control Models in Time Series [1.9371782627708491]
We propose a new high dimensional Bayesian Optimization algorithm combining Recurrent neural networks.
The proposed RNN-BO algorithm can solve the optimal control problems in the lower dimension space.
We also discuss the impacts of different numbers of the RNN layers and training epochs on the trade-off between solution quality and related computational efforts.
arXiv Detail & Related papers (2022-01-01T08:40:17Z) - Joint Deep Reinforcement Learning and Unfolding: Beam Selection and
Precoding for mmWave Multiuser MIMO with Lens Arrays [54.43962058166702]
millimeter wave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems with discrete lens arrays have received great attention.
In this work, we investigate the joint design of a beam precoding matrix for mmWave MU-MIMO systems with DLA.
arXiv Detail & Related papers (2021-01-05T03:55:04Z) - Data-Driven Random Access Optimization in Multi-Cell IoT Networks with
NOMA [78.60275748518589]
Non-orthogonal multiple access (NOMA) is a key technology to enable massive machine type communications (mMTC) in 5G networks and beyond.
In this paper, NOMA is applied to improve the random access efficiency in high-density spatially-distributed multi-cell wireless IoT networks.
A novel formulation of random channel access management is proposed, in which the transmission probability of each IoT device is tuned to maximize the geometric mean of users' expected capacity.
arXiv Detail & Related papers (2021-01-02T15:21:08Z) - Multi-Level Evolution Strategies for High-Resolution Black-Box Control [0.2320417845168326]
This paper introduces a multi-level (m-lev) mechanism into Evolution Strategies (ESs)
It addresses a class of global optimization problems that could benefit from fine discretization of their decision variables.
arXiv Detail & Related papers (2020-10-04T09:24:40Z) - Learning High-Level Policies for Model Predictive Control [54.00297896763184]
Model Predictive Control (MPC) provides robust solutions to robot control tasks.
We propose a self-supervised learning algorithm for learning a neural network high-level policy.
We show that our approach can handle situations that are difficult for standard MPC.
arXiv Detail & Related papers (2020-07-20T17:12:34Z) - Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding
Design for Multiuser MIMO Systems [59.804810122136345]
We propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed.
An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed.
We show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.
arXiv Detail & Related papers (2020-06-15T02:57:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.