Neural Time-Reversed Generalized Riccati Equation

• URL: http://arxiv.org/abs/2312.09310v1
• Date: Thu, 14 Dec 2023 19:29:37 GMT
• Title: Neural Time-Reversed Generalized Riccati Equation
• Authors: Alessandro Betti, Michele Casoni, Marco Gori, Simone Marullo, Stefano Melacci, Matteo Tiezzi
• Abstract summary: Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time.
• Score: 60.92253836775246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.

Related papers

• Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems [13.285775352653546]
  This paper proposes a system of neural differential equations representing first- and second-order score functions along trajectories based on deep neural networks. We reformulate the mean viscous field control (MFC) problem with individual noises into an unconstrained optimization problem framed by the proposed neural ODE system.
  arXiv  Detail & Related papers  (2024-09-24T21:45:55Z)
• The cost of solving linear differential equations on a quantum computer: fast-forwarding to explicit resource counts [0.0]
  We give the first non-asymptotic computation of the cost of encoding the solution to general linear ordinary differential equations into quantum states. We show that the stability properties of a large class of classical dynamics allow their fast-forwarding. We find that the history state can always be output with complexity $O(T1/2)$ for any stable linear system.
  arXiv  Detail & Related papers  (2023-09-14T17:25:43Z)
• Neural ODEs as Feedback Policies for Nonlinear Optimal Control [1.8514606155611764]
  We use Neural ordinary differential equations (Neural ODEs) to model continuous time dynamics as differential equations parametrized with neural networks. We propose the use of a neural control policy posed as a Neural ODE to solve general nonlinear optimal control problems.
  arXiv  Detail & Related papers  (2022-10-20T13:19:26Z)
• Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control [27.38625075499457]
  We present a scalable deep learning approach to solve opinion dynamics optimal control problems with mean field term coupling in the dynamics and cost function. The proposed framework opens up the possibility for future applications on extremely large-scale problems.
  arXiv  Detail & Related papers  (2022-04-05T22:07:32Z)
• Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization [50.83356836818667]
  gradient Langevin Dynamics is one of the most fundamental algorithms to solve non-eps optimization problems. In this paper, we show two variants of this kind, namely the Variance Reduced Langevin Dynamics and the Recursive Gradient Langevin Dynamics.
  arXiv  Detail & Related papers  (2022-03-30T11:39:00Z)
• Continuous-Time Fitted Value Iteration for Robust Policies [93.25997466553929]
  Solving the Hamilton-Jacobi-Bellman equation is important in many domains including control, robotics and economics. We propose continuous fitted value iteration (cFVI) and robust fitted value iteration (rFVI) These algorithms leverage the non-linear control-affine dynamics and separable state and action reward of many continuous control problems.
  arXiv  Detail & Related papers  (2021-10-05T11:33:37Z)
• Robust Value Iteration for Continuous Control Tasks [99.00362538261972]
  When transferring a control policy from simulation to a physical system, the policy needs to be robust to variations in the dynamics to perform well. We present Robust Fitted Value Iteration, which uses dynamic programming to compute the optimal value function on the compact state domain. We show that robust value is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.
  arXiv  Detail & Related papers  (2021-05-25T19:48:35Z)
• Meta-Solver for Neural Ordinary Differential Equations [77.8918415523446]
  We investigate how the variability in solvers' space can improve neural ODEs performance. We show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks.
  arXiv  Detail & Related papers  (2021-03-15T17:26:34Z)
• Direct Optimal Control Approach to Laser-Driven Quantum Particle Dynamics [77.34726150561087]
  We propose direct optimal control as a robust and flexible alternative to indirect control theory. The method is illustrated for the case of laser-driven wavepacket dynamics in a bistable potential.
  arXiv  Detail & Related papers  (2020-10-08T07:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.