A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEs

• URL: http://arxiv.org/abs/2204.05796v1
• Date: Tue, 12 Apr 2022 13:31:19 GMT
• Title: A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEs
• Authors: Shaolin Ji, Shige Peng, Ying Peng and Xichuan Zhang
• Abstract summary: In this paper, we focus on the numerical solution of high-dimensional optimal control problem driven by fully-coupled forward-backward differential equations (FBSDEs in short) through deep learning. We first transform the problem into a Stackelberg differential game(leader-follower problem), then a cross-optimization method (COCO method) is developed where the leader's cost functional and the follower's cost are optimized via deep neural networks. As for the numerical results, we compute two examples of the investment-consumption problem solved through utility models, and the results of both examples demonstrate the effectiveness of our
• Score: 0.2064612766965483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we mainly focus on the numerical solution of high-dimensional stochastic optimal control problem driven by fully-coupled forward-backward stochastic differential equations (FBSDEs in short) through deep learning. We first transform the problem into a stochastic Stackelberg differential game(leader-follower problem), then a cross-optimization method (CO method) is developed where the leader's cost functional and the follower's cost functional are optimized alternatively via deep neural networks. As for the numerical results, we compute two examples of the investment-consumption problem solved through stochastic recursive utility models, and the results of both examples demonstrate the effectiveness of our proposed algorithm.

Related papers

• A Guide to Stochastic Optimisation for Large-Scale Inverse Problems [4.926711494319977]
  optimisation algorithms are the de facto standard for machine learning with large amounts of data. We provide a comprehensive account of the state-of-the-art in optimisation from the viewpoint of inverse problems. We focus on the challenges for optimisation that are unique and are not commonly encountered in machine learning.
  arXiv  Detail & Related papers  (2024-06-10T15:02:30Z)
• Stochastic Q-learning for Large Discrete Action Spaces [79.1700188160944]
  In complex environments with discrete action spaces, effective decision-making is critical in reinforcement learning (RL) We present value-based RL approaches which, as opposed to optimizing over the entire set of $n$ actions, only consider a variable set of actions, possibly as small as $mathcalO(log(n)$)$. The presented value-based RL methods include, among others, Q-learning, StochDQN, StochDDQN, all of which integrate this approach for both value-function updates and action selection.
  arXiv  Detail & Related papers  (2024-05-16T17:58:44Z)
• Analyzing and Enhancing the Backward-Pass Convergence of Unrolled Optimization [50.38518771642365]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which often lacks a closed form. This paper provides theoretical insights into the backward pass of unrolled optimization, showing that it is equivalent to the solution of a linear system by a particular iterative method. A system called Folded Optimization is proposed to construct more efficient backpropagation rules from unrolled solver implementations.
  arXiv  Detail & Related papers  (2023-12-28T23:15:18Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• Stochastic Unrolled Federated Learning [85.6993263983062]
  We introduce UnRolled Federated learning (SURF), a method that expands algorithm unrolling to federated learning. Our proposed method tackles two challenges of this expansion, namely the need to feed whole datasets to the unrolleds and the decentralized nature of federated learning.
  arXiv  Detail & Related papers  (2023-05-24T17:26:22Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• Deep Learning for Constrained Utility Maximisation [0.0]
  This paper proposes two algorithms for solving control problems with deep learning. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman equation. The second uses the full power of the duality method to solve non-Markovian problems.
  arXiv  Detail & Related papers  (2020-08-26T18:40:57Z)
• Combining Deep Learning and Optimization for Security-Constrained Optimal Power Flow [94.24763814458686]
  Security-constrained optimal power flow (SCOPF) is fundamental in power systems. Modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs. This paper proposes a novel approach that combines deep learning and robust optimization techniques.
  arXiv  Detail & Related papers  (2020-07-14T12:38:21Z)
• Solving stochastic optimal control problem via stochastic maximum principle with deep learning method [0.2064612766965483]
  Three algorithms are proposed to solve the new control problem. An important application of this method is to calculate the sub-linear expectations, which correspond to a kind of fully nonlinear PDEs.
  arXiv  Detail & Related papers  (2020-07-05T02:28:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.