Recent Developments in Machine Learning Methods for Stochastic Control and Games

• URL: http://arxiv.org/abs/2303.10257v3
• Date: Mon, 11 Mar 2024 04:59:57 GMT
• Title: Recent Developments in Machine Learning Methods for Stochastic Control and Games
• Authors: Ruimeng Hu, Mathieu Lauri\`ere
• Abstract summary: Recently, computational methods based on machine learning have been developed for solving control problems and games. We focus on deep learning methods that have unlocked the possibility of solving such problems, even in high dimensions or when the structure is very complex. This paper provides an introduction to these methods and summarizes the state-of-the-art works at the crossroad of machine learning and control and games.
• Score: 3.3993877661368757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stochastic optimal control and games have a wide range of applications, from finance and economics to social sciences, robotics, and energy management. Many real-world applications involve complex models that have driven the development of sophisticated numerical methods. Recently, computational methods based on machine learning have been developed for solving stochastic control problems and games. In this review, we focus on deep learning methods that have unlocked the possibility of solving such problems, even in high dimensions or when the structure is very complex, beyond what traditional numerical methods can achieve. We consider mostly the continuous time and continuous space setting. Many of the new approaches build on recent neural-network-based methods for solving high-dimensional partial differential equations or backward stochastic differential equations, or on model-free reinforcement learning for Markov decision processes that have led to breakthrough results. This paper provides an introduction to these methods and summarizes the state-of-the-art works at the crossroad of machine learning and stochastic control and games.

Related papers

• Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems [0.0]
  reinforcement learning has been applied to decision-making in several applications, most notably in games. We introduce two novel reward functions to drive the flow of the transported field. Results show that certain controls can be implemented successfully in these applications.
  arXiv  Detail & Related papers  (2024-02-22T11:06:07Z)
• Machine Learning Insides OptVerse AI Solver: Design Principles and Applications [74.67495900436728]
  We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
  arXiv  Detail & Related papers  (2024-01-11T15:02:15Z)
• Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms [3.222802562733787]
  We propose a numerical methodology for finding the closed-loop Nash equilibrium of delay differential games through deep learning. These games are prevalent in finance and economics where multi-agent interaction and delayed effects are often desired features in a model.
  arXiv  Detail & Related papers  (2023-07-12T21:02:45Z)
• Stop overkilling simple tasks with black-box models and use transparent models instead [57.42190785269343]
  Deep learning approaches are able to extract features autonomously from raw data. This allows for bypassing the feature engineering process. Deep learning strategies often outperform traditional models in terms of accuracy.
  arXiv  Detail & Related papers  (2023-02-06T14:28:49Z)
• On Robust Numerical Solver for ODE via Self-Attention Mechanism [82.95493796476767]
  We explore training efficient and robust AI-enhanced numerical solvers with a small data size by mitigating intrinsic noise disturbances. We first analyze the ability of the self-attention mechanism to regulate noise in supervised learning and then propose a simple-yet-effective numerical solver, Attr, which introduces an additive self-attention mechanism to the numerical solution of differential equations.
  arXiv  Detail & Related papers  (2023-02-05T01:39:21Z)
• Neural Operator: Is data all you need to model the world? An insight into the impact of Physics Informed Machine Learning [13.050410285352605]
  We provide an insight into how data-driven approaches can complement conventional techniques to solve engineering and physics problems. We highlight a novel and fast machine learning-based approach to learning the solution operator of a PDE operator learning.
  arXiv  Detail & Related papers  (2023-01-30T23:29:33Z)
• Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics [77.34726150561087]
  Recent developments in artificial neural networks, particularly deep learning (DL), are reviewed in detail. Both hybrid and pure machine learning (ML) methods are discussed. History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics.
  arXiv  Detail & Related papers  (2022-12-18T02:03:00Z)
• Gradients are Not All You Need [28.29420710601308]
  We discuss a common chaos based failure mode which appears in a variety of differentiable circumstances. We trace this failure to the spectrum of the Jacobian of the system under study, and provide criteria for when a practitioner might expect this failure to spoil their differentiation based optimization algorithms.
  arXiv  Detail & Related papers  (2021-11-10T16:51:04Z)
• IQ-Learn: Inverse soft-Q Learning for Imitation [95.06031307730245]
  imitation learning from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics. Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence. We introduce a method for dynamics-aware IL which avoids adversarial training by learning a single Q-function.
  arXiv  Detail & Related papers  (2021-06-23T03:43:10Z)
• Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks [15.276951055528237]
  We review the state-of-the-art methods that allow continuous learning of computational models over time. We focus on the learning algorithms that perform continuous learning in an online fashion from considerably large (or infinite) sequential data. We critically analyze the key challenges associated with continual learning for autonomous real-world systems.
  arXiv  Detail & Related papers  (2021-05-26T07:38:20Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.