Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

• URL: http://arxiv.org/abs/2110.11265v2
• Date: Sat, 23 Oct 2021 23:02:20 GMT
• Title: Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations
• Authors: Erfan Pirmorad, Faraz Khoshbakhtian, Farnam Mansouri, Amir-massoud Farahmand
• Abstract summary: We formulate the problem of controlling partial differential equations as a reinforcement learning problem. We present a learning-based, distributed control approach for online control of a system of SPDEs with high dimensional state-action space.
• Score: 10.746602033809943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many areas, such as the physical sciences, life sciences, and finance, control approaches are used to achieve a desired goal in complex dynamical systems governed by differential equations. In this work we formulate the problem of controlling stochastic partial differential equations (SPDE) as a reinforcement learning problem. We present a learning-based, distributed control approach for online control of a system of SPDEs with high dimensional state-action space using deep deterministic policy gradient method. We tested the performance of our method on the problem of controlling the stochastic Burgers' equation, describing a turbulent fluid flow in an infinitely large domain.

Related papers

• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• A Simulation-Free Deep Learning Approach to Stochastic Optimal Control [12.699529713351287]
  We propose a simulation-free algorithm for the solution of generic problems in optimal control (SOC) Unlike existing methods, our approach does not require the solution of an adjoint problem.
  arXiv  Detail & Related papers  (2024-10-07T16:16:53Z)
• DeepSPoC: A Deep Learning-Based PDE Solver Governed by Sequential Propagation of Chaos [7.808454074695533]
  Sequential propagation of chaos (SPoC) is a recently developed tool to solve mean-field differential equations. We present a new method (deepSPoC) that combines the interacting particle system of SPoC and deep learning. For high-dimensional problems, spatial adaptive method are designed to further improve the accuracy and efficiency of deepSPoC.
  arXiv  Detail & Related papers  (2024-08-29T10:02:29Z)
• KEEC: Koopman Embedded Equivariant Control [29.738391644702947]
  An efficient way to control systems with unknown nonlinear dynamics is to find an appropriate embedding or representation. Koopman Embedded Equivariant Control (KEEC) learns an embedding of the states and vector fields such that a Koopman operator is approximated as the latent dynamics. Our algorithm achieves superior performances in the experiments conducted on various control domains.
  arXiv  Detail & Related papers  (2023-12-04T00:11:27Z)
• GRAPE optimization for open quantum systems with time-dependent decoherence rates driven by coherent and incoherent controls [77.34726150561087]
  The GRadient Ascent Pulse Engineering (GRAPE) method is widely used for optimization in quantum control. We adopt GRAPE method for optimizing objective functionals for open quantum systems driven by both coherent and incoherent controls. The efficiency of the algorithm is demonstrated through numerical simulations for the state-to-state transition problem.
  arXiv  Detail & Related papers  (2023-07-17T13:37:18Z)
• Semi-supervised Learning of Partial Differential Operators and Dynamical Flows [68.77595310155365]
  We present a novel method that combines a hyper-network solver with a Fourier Neural Operator architecture. We test our method on various time evolution PDEs, including nonlinear fluid flows in one, two, and three spatial dimensions. The results show that the new method improves the learning accuracy at the time point of supervision point, and is able to interpolate and the solutions to any intermediate time.
  arXiv  Detail & Related papers  (2022-07-28T19:59:14Z)
• Physics-constrained Unsupervised Learning of Partial Differential Equations using Meshes [1.066048003460524]
  Graph neural networks show promise in accurately representing irregularly meshed objects and learning their dynamics. In this work, we represent meshes naturally as graphs, process these using Graph Networks, and formulate our physics-based loss to provide an unsupervised learning framework for partial differential equations (PDE) Our framework will enable the application of PDE solvers in interactive settings, such as model-based control of soft-body deformations.
  arXiv  Detail & Related papers  (2022-03-30T19:22:56Z)
• Deep Learning Approximation of Diffeomorphisms via Linear-Control Systems [91.3755431537592]
  We consider a control system of the form $dot x = sum_i=1lF_i(x)u_i$, with linear dependence in the controls. We use the corresponding flow to approximate the action of a diffeomorphism on a compact ensemble of points.
  arXiv  Detail & Related papers  (2021-10-24T08:57:46Z)
• DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimization [77.34726150561087]
  We introduce Discovery of Dynamical Systems via Moving Horizon Optimization (DySMHO), a scalable machine learning framework. DySMHO sequentially learns the underlying governing equations from a large dictionary of basis functions. Canonical nonlinear dynamical system examples are used to demonstrate that DySMHO can accurately recover the governing laws.
  arXiv  Detail & Related papers  (2021-07-30T20:35:03Z)
• Learning to Control PDEs with Differentiable Physics [102.36050646250871]
  We present a novel hierarchical predictor-corrector scheme which enables neural networks to learn to understand and control complex nonlinear physical systems over long time frames. We demonstrate that our method successfully develops an understanding of complex physical systems and learns to control them for tasks involving PDEs.
  arXiv  Detail & Related papers  (2020-01-21T11:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.