Adaptive deep density approximation for stochastic dynamical systems

• URL: http://arxiv.org/abs/2405.02810v1
• Date: Sun, 5 May 2024 04:29:22 GMT
• Title: Adaptive deep density approximation for stochastic dynamical systems
• Authors: Junjie He, Qifeng Liao, Xiaoliang Wan,
• Abstract summary: A new temporal KRnet is proposed to approximate the probability density functions (PDFs) iteration of the state variables. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function. A temporal decomposition technique is also employed to improve the long-time integration.
• Score: 0.5120567378386615
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper we consider adaptive deep neural network approximation for stochastic dynamical systems. Based on the Liouville equation associated with the stochastic dynamical systems, a new temporal KRnet (tKRnet) is proposed to approximate the probability density functions (PDFs) of the state variables. The tKRnet gives an explicit density model for the solution of the Liouville equation, which alleviates the curse of dimensionality issue that limits the application of traditional grid based numerical methods. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function, where samples are generated iteratively using the approximate density function at each iteration. A temporal decomposition technique is also employed to improve the long-time integration. Theoretical analysis of our proposed method is provided, and numerical examples are presented to demonstrate its performance.

Related papers

• A Fokker-Planck-Based Loss Function that Bridges Dynamics with Density Estimation [1.8434042562191815]
  We derive a novel loss function from the Fokker-Planck equation that links dynamical system models with their probability density functions. For density estimation, we propose a density estimator that integrates a Gaussian Mixture Model with a normalizing flow model. It is compatible with a variety of data-based training methodologies, including maximum likelihood and score matching.
  arXiv  Detail & Related papers  (2025-02-24T22:27:25Z)
• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• DynGMA: a robust approach for learning stochastic differential equations from data [13.858051019755283]
  We introduce novel approximations to the transition density of the parameterized SDE. Our method exhibits superior accuracy compared to baseline methods in learning the fully unknown drift diffusion functions. It is capable of handling data with low time resolution and variable, even uncontrollable, time step sizes.
  arXiv  Detail & Related papers  (2024-02-22T12:09:52Z)
• Adaptive importance sampling for Deep Ritz [7.123920027048777]
  We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations (PDEs) One network is employed to approximate the solution of PDEs, while the other one is a deep generative model used to generate new collocation points to refine the training set. Compared to the original Deep Ritz method, the proposed adaptive method improves accuracy, especially for problems characterized by low regularity and high dimensionality.
  arXiv  Detail & Related papers  (2023-10-26T06:35:08Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• DAS: A deep adaptive sampling method for solving partial differential equations [2.934397685379054]
  We propose a deep adaptive sampling (DAS) method for solving partial differential equations (PDEs) Deep neural networks are utilized to approximate the solutions of PDEs and deep generative models are employed to generate new collocation points that refine the training set. We present a theoretical analysis to show that the proposed DAS method can reduce the error bound and demonstrate its effectiveness with numerical experiments.
  arXiv  Detail & Related papers  (2021-12-28T08:37:47Z)
• Adaptive deep density approximation for Fokker-Planck equations [0.0]
  We present a novel deep density approximation strategy based on KRnet (ADDAKR) for solving the steady-state Fokker-Planck equation. We show that KRnet can efficiently estimate general high-dimensional density functions.
  arXiv  Detail & Related papers  (2021-03-20T13:49:52Z)
• LocalDrop: A Hybrid Regularization for Deep Neural Networks [98.30782118441158]
  We propose a new approach for the regularization of neural networks by the local Rademacher complexity called LocalDrop. A new regularization function for both fully-connected networks (FCNs) and convolutional neural networks (CNNs) has been developed based on the proposed upper bound of the local Rademacher complexity.
  arXiv  Detail & Related papers  (2021-03-01T03:10:11Z)
• A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions [17.14287157979558]
  We propose an adaptively weighted gradient Langevin dynamics (SGLD) for learning in big data statistics. The proposed algorithm is tested on benchmark datasets including CIFAR100.
  arXiv  Detail & Related papers  (2020-10-19T19:20:47Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.