Solving a class of stochastic optimal control problems by physics-informed neural networks

• URL: http://arxiv.org/abs/2402.15592v2
• Date: Fri, 20 Jun 2025 12:18:30 GMT
• Title: Solving a class of stochastic optimal control problems by physics-informed neural networks
• Authors: Zhe Jiao, Wantao Jia, Weiqiu Zhu,
• Abstract summary: The aim of this work is to develop a deep learning method for solving high-dimensional control problems based on the Hamilton-Jacobi-Bellman (HJB) equation and physics-informed learning.<n>Our approach is to parameterize the feedback control and the value function using a decoupled neural network with multiple outputs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The aim of this work is to develop a deep learning method for solving high-dimensional stochastic control problems based on the Hamilton--Jacobi--Bellman (HJB) equation and physics-informed learning. Our approach is to parameterize the feedback control and the value function using a decoupled neural network with multiple outputs. We train this network by using a loss function with penalty terms that enforce the HJB equation along the sampled trajectories generated by the controlled system. More significantly, numerical results on various applications are carried out to demonstrate that the proposed approach is efficient and applicable.

Related papers

• Neural Hamiltonian Operator [2.1756081703276]
  An alternative to traditional dynamic programming is Pontryagin's Maximum Principle (PMP), which recasts the problem as a system of Forward-Backward Differential Equations (FBSDEs)<n>In this paper, we introduce a formal framework for solving such problems with deep learning by defining a textbfNeural Hamiltonian Operator (NHO).<n>We show how the optimal NHO can be found by training the underlying networks to enforce the consistency conditions dictated by the PMP.
  arXiv  Detail & Related papers  (2025-07-02T02:56:49Z)
• Deep Learning for Continuous-time Stochastic Control with Jumps [1.6112718683989882]
  We introduce a model-based deep-learning approach to solve finite-horizon continuous-time control problems with jumps.<n>We iteratively train two neural networks: one to represent the optimal policy and the other to approximate the value function.
  arXiv  Detail & Related papers  (2025-05-21T14:57:39Z)
• Integrating Physics-Informed Deep Learning and Numerical Methods for Robust Dynamics Discovery and Parameter Estimation [0.0]
  In this work, we combine deep learning techniques and classic numerical methods for differential equations to solve two challenging problems in dynamical systems theory. Results demonstrate the effectiveness of the proposed approaches on a suite of test problems exhibiting chaotic dynamics.
  arXiv  Detail & Related papers  (2024-10-05T22:40:02Z)
• Formally Verified Physics-Informed Neural Control Lyapunov Functions [4.2162963332651575]
  Control Lyapunov functions are a central tool in the design and analysis of stabilizing controllers for nonlinear systems. In this paper, we investigate physics-informed learning and formal verification of neural network control Lyapunov functions.
  arXiv  Detail & Related papers  (2024-09-30T17:27:56Z)
• Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics [8.528817025440746]
  This paper introduces a novel Gaussian Process (GP) based neural operator for solving parametric differential equations. We propose a neural operator-embedded kernel'' wherein the GP kernel is formulated in the latent space learned using a neural operator. Our results highlight the efficacy of this framework in solving complex PDEs while maintaining robustness in uncertainty estimation.
  arXiv  Detail & Related papers  (2024-09-17T08:12:38Z)
• Hamilton-Jacobi Based Policy-Iteration via Deep Operator Learning [9.950128864603599]
  We incorporate DeepONet with a recently developed policy scheme to numerically solve optimal control problems. A notable feature of our approach is that once the neural network is trained, the solution to the optimal control problem and HJB equations can be inferred quickly.
  arXiv  Detail & Related papers  (2024-06-16T12:53:17Z)
• Solving Poisson Equations using Neural Walk-on-Spheres [80.1675792181381]
  We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. We demonstrate the superiority of NWoS in accuracy, speed, and computational costs.
  arXiv  Detail & Related papers  (2024-06-05T17:59:22Z)
• Deep multitask neural networks for solving some stochastic optimal control problems [0.0]
  In this paper, we consider a class of optimal control problems and introduce an effective solution employing neural networks. To train our multitask neural network, we introduce a novel scheme that dynamically balances the learning across tasks. Through numerical experiments on real-world derivatives pricing problems, we prove that our method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-01-23T17:20:48Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networks [0.0]
  We propose a multi-level algorithm for the solution of nonlinear optimization problems and analyze its evaluation complexity. We apply it to the solution of partial differential equations using physics-informed neural networks (PINNs) and show on a few test problems that the approach results in better solutions and significant computational savings.
  arXiv  Detail & Related papers  (2023-05-23T19:12:02Z)
• Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RL [106.82295532402335]
  Existing reinforcement learning algorithms suffer from computational intractability, strong statistical assumptions, and suboptimal sample complexity. We provide the first computationally efficient algorithm that attains rate-optimal sample complexity with respect to the desired accuracy level. Our algorithm, MusIK, combines systematic exploration with representation learning based on multi-step inverse kinematics.
  arXiv  Detail & Related papers  (2023-04-12T14:51:47Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Experimental study of Neural ODE training with adaptive solver for dynamical systems modeling [72.84259710412293]
  Some ODE solvers called adaptive can adapt their evaluation strategy depending on the complexity of the problem at hand. This paper describes a simple set of experiments to show why adaptive solvers cannot be seamlessly leveraged as a black-box for dynamical systems modelling.
  arXiv  Detail & Related papers  (2022-11-13T17:48:04Z)
• Accelerating the training of single-layer binary neural networks using the HHL quantum algorithm [58.720142291102135]
  We show that useful information can be extracted from the quantum-mechanical implementation of Harrow-Hassidim-Lloyd (HHL) This paper shows, however, that useful information can be extracted from the quantum-mechanical implementation of HHL, and used to reduce the complexity of finding the solution on the classical side.
  arXiv  Detail & Related papers  (2022-10-23T11:58:05Z)
• Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control [27.38625075499457]
  We present a scalable deep learning approach to solve opinion dynamics optimal control problems with mean field term coupling in the dynamics and cost function. The proposed framework opens up the possibility for future applications on extremely large-scale problems.
  arXiv  Detail & Related papers  (2022-04-05T22:07:32Z)
• Amortized Implicit Differentiation for Stochastic Bilevel Optimization [53.12363770169761]
  We study a class of algorithms for solving bilevel optimization problems in both deterministic and deterministic settings. We exploit a warm-start strategy to amortize the estimation of the exact gradient. By using this framework, our analysis shows these algorithms to match the computational complexity of methods that have access to an unbiased estimate of the gradient.
  arXiv  Detail & Related papers  (2021-11-29T15:10:09Z)
• ES-Based Jacobian Enables Faster Bilevel Optimization [53.675623215542515]
  Bilevel optimization (BO) has arisen as a powerful tool for solving many modern machine learning problems. Existing gradient-based methods require second-order derivative approximations via Jacobian- or/and Hessian-vector computations. We propose a novel BO algorithm, which adopts Evolution Strategies (ES) based method to approximate the response Jacobian matrix in the hypergradient of BO.
  arXiv  Detail & Related papers  (2021-10-13T19:36:50Z)
• Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms [71.62575565990502]
  We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure. We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
  arXiv  Detail & Related papers  (2021-06-09T08:05:36Z)
• Logistic Q-Learning [87.00813469969167]
  We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The main feature of our algorithm is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error.
  arXiv  Detail & Related papers  (2020-10-21T17:14:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.