Solving stochastic optimal control problem via stochastic maximum principle with deep learning method

• URL: http://arxiv.org/abs/2007.02227v5
• Date: Tue, 22 Jun 2021 02:39:09 GMT
• Title: Solving stochastic optimal control problem via stochastic maximum principle with deep learning method
• Authors: Shaolin Ji, Shige Peng, Ying Peng, Xichuan Zhang
• Abstract summary: Three algorithms are proposed to solve the new control problem. An important application of this method is to calculate the sub-linear expectations, which correspond to a kind of fully nonlinear PDEs.
• Score: 0.2064612766965483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we aim to solve the high dimensional stochastic optimal control problem from the view of the stochastic maximum principle via deep learning. By introducing the extended Hamiltonian system which is essentially an FBSDE with a maximum condition, we reformulate the original control problem as a new one. Three algorithms are proposed to solve the new control problem. Numerical results for different examples demonstrate the effectiveness of our proposed algorithms, especially in high dimensional cases. And an important application of this method is to calculate the sub-linear expectations, which correspond to a kind of fully nonlinear PDEs.

Related papers

• A Simulation-Free Deep Learning Approach to Stochastic Optimal Control [12.699529713351287]
  We propose a simulation-free algorithm for the solution of generic problems in optimal control (SOC) Unlike existing methods, our approach does not require the solution of an adjoint problem.
  arXiv  Detail & Related papers  (2024-10-07T16:16:53Z)
• Hamilton-Jacobi Based Policy-Iteration via Deep Operator Learning [9.950128864603599]
  We incorporate DeepONet with a recently developed policy scheme to numerically solve optimal control problems. A notable feature of our approach is that once the neural network is trained, the solution to the optimal control problem and HJB equations can be inferred quickly.
  arXiv  Detail & Related papers  (2024-06-16T12:53:17Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEs [1.0703175070560689]
  We first transform the problem into a Stackelberg differential game problem (leader-follower problem) We compute two examples of the investment-consumption problem solved through utility models. The results of both examples demonstrate the effectiveness of our proposed algorithm.
  arXiv  Detail & Related papers  (2022-04-12T13:31:19Z)
• A control method for solving high-dimensional Hamiltonian systems through deep neural networks [0.2752817022620644]
  We first introduce a corresponding optimal control problem such that the Hamiltonian system of control problem is exactly what we need to solve, then develop two different algorithms suitable for different cases of the control problem and approximate the control via deep neural networks. From the numerical results, comparing with the Deep FBSDE method which was developed previously from the view of solving FBSDEs, the novel algorithms converge faster, which means that they require fewer training steps, and demonstrate more stable convergences for different Hamiltonian systems.
  arXiv  Detail & Related papers  (2021-11-04T05:22:08Z)
• Deep Learning Approximation of Diffeomorphisms via Linear-Control Systems [91.3755431537592]
  We consider a control system of the form $dot x = sum_i=1lF_i(x)u_i$, with linear dependence in the controls. We use the corresponding flow to approximate the action of a diffeomorphism on a compact ensemble of points.
  arXiv  Detail & Related papers  (2021-10-24T08:57:46Z)
• Logistic Q-Learning [87.00813469969167]
  We propose a new reinforcement learning algorithm derived from a regularized linear-programming formulation of optimal control in MDPs. The main feature of our algorithm is a convex loss function for policy evaluation that serves as a theoretically sound alternative to the widely used squared Bellman error.
  arXiv  Detail & Related papers  (2020-10-21T17:14:31Z)
• Deep Learning for Constrained Utility Maximisation [0.0]
  This paper proposes two algorithms for solving control problems with deep learning. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman equation. The second uses the full power of the duality method to solve non-Markovian problems.
  arXiv  Detail & Related papers  (2020-08-26T18:40:57Z)
• Follow the bisector: a simple method for multi-objective optimization [65.83318707752385]
  We consider optimization problems, where multiple differentiable losses have to be minimized. The presented method computes descent direction in every iteration to guarantee equal relative decrease of objective functions.
  arXiv  Detail & Related papers  (2020-07-14T09:50:33Z)
• Conditional gradient methods for stochastically constrained convex minimization [54.53786593679331]
  We propose two novel conditional gradient-based methods for solving structured convex optimization problems. The most important feature of our framework is that only a subset of the constraints is processed at each iteration. Our algorithms rely on variance reduction and smoothing used in conjunction with conditional gradient steps, and are accompanied by rigorous convergence guarantees.
  arXiv  Detail & Related papers  (2020-07-07T21:26:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.