Data-driven initialization of deep learning solvers for Hamilton-Jacobi-Bellman PDEs

• URL: http://arxiv.org/abs/2207.09299v1
• Date: Tue, 19 Jul 2022 14:34:07 GMT
• Title: Data-driven initialization of deep learning solvers for Hamilton-Jacobi-Bellman PDEs
• Authors: Anastasia Borovykh, Dante Kalise, Alexis Laignelet, Panos Parpas
• Abstract summary: A state-dependent Riccati equation control law is first used to generate a gradient-augmented synthetic dataset for supervised learning. The resulting model becomes a warm start for the minimization of a loss function based on the residual of the HJB PDE.
• Score: 3.249853429482705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A deep learning approach for the approximation of the Hamilton-Jacobi-Bellman partial differential equation (HJB PDE) associated to the Nonlinear Quadratic Regulator (NLQR) problem. A state-dependent Riccati equation control law is first used to generate a gradient-augmented synthetic dataset for supervised learning. The resulting model becomes a warm start for the minimization of a loss function based on the residual of the HJB PDE. The combination of supervised learning and residual minimization avoids spurious solutions and mitigate the data inefficiency of a supervised learning-only approach. Numerical tests validate the different advantages of the proposed methodology.

Related papers

• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Active Learning for Neural PDE Solvers [18.665448858377694]
  Active Learning could help surrogate models reach the same accuracy with smaller training sets. We introduce AL4PDE, a modular and active learning benchmark. We show that AL reduces the average error by up to 71% compared to random sampling.
  arXiv  Detail & Related papers  (2024-08-02T18:48:58Z)
• Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning [1.8175282137722093]
  Uncertainty (UQ) in scientific machine learning (SciML) combines the powerful predictive power of SciML with methods for quantifying the reliability of the learned models. We provide a new interpretation for UQ problems by establishing a new theoretical connection between some Bayesian inference problems arising in SciML and viscous Hamilton-Jacobi partial differential equations (HJ PDEs) We develop a new Riccati-based methodology that provides computational advantages when continuously updating the model predictions.
  arXiv  Detail & Related papers  (2024-04-12T20:54:01Z)
• Leveraging Hamilton-Jacobi PDEs with time-dependent Hamiltonians for continual scientific machine learning [1.8175282137722093]
  We address two major challenges in scientific machine learning (SciML) We establish a new theoretical connection between optimization problems arising from SciML and a generalized Hopf formula. Existing HJ PDE solvers and optimal control algorithms can be reused to design new efficient training approaches.
  arXiv  Detail & Related papers  (2023-11-13T22:55:56Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity [51.476337785345436]
  We study a pessimistic variant of Q-learning in the context of finite-horizon Markov decision processes. A variance-reduced pessimistic Q-learning algorithm is proposed to achieve near-optimal sample complexity.
  arXiv  Detail & Related papers  (2022-02-28T15:39:36Z)
• Learning via nonlinear conjugate gradients and depth-varying neural ODEs [5.565364597145568]
  The inverse problem of supervised reconstruction of depth-variable parameters in a neural ordinary differential equation (NODE) is considered. The proposed parameter reconstruction is done for a general first order differential equation by minimizing a cost functional. The sensitivity problem can estimate changes in the network output under perturbation of the trained parameters.
  arXiv  Detail & Related papers  (2022-02-11T17:00:48Z)
• Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of Partial Differential Equations [5.584060970507507]
  A Parsimony Enhanced Sparse Bayesian Learning (PeSBL) method is developed for discovering the governing Partial Differential Equations (PDEs) of nonlinear dynamical systems. Results of numerical case studies indicate that the governing PDEs of many canonical dynamical systems can be correctly identified using the proposed PeSBL method.
  arXiv  Detail & Related papers  (2021-07-08T00:56:11Z)
• Logistic Q-Learning [87.00813469969167]
  We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The main feature of our algorithm is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error.
  arXiv  Detail & Related papers  (2020-10-21T17:14:31Z)
• Combining Deep Learning and Optimization for Security-Constrained Optimal Power Flow [94.24763814458686]
  Security-constrained optimal power flow (SCOPF) is fundamental in power systems. Modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs. This paper proposes a novel approach that combines deep learning and robust optimization techniques.
  arXiv  Detail & Related papers  (2020-07-14T12:38:21Z)
• Strictly Batch Imitation Learning by Energy-based Distribution Matching [104.33286163090179]
  Consider learning a policy purely on the basis of demonstrated behavior -- that is, with no access to reinforcement signals, no knowledge of transition dynamics, and no further interaction with the environment. One solution is simply to retrofit existing algorithms for apprenticeship learning to work in the offline setting. But such an approach leans heavily on off-policy evaluation or offline model estimation, and can be indirect and inefficient. We argue that a good solution should be able to explicitly parameterize a policy, implicitly learn from rollout dynamics, and operate in an entirely offline fashion.
  arXiv  Detail & Related papers  (2020-06-25T03:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.