Learning Stochastic Nonlinear Dynamics with Embedded Latent Transfer Operators

• URL: http://arxiv.org/abs/2501.02721v3
• Date: Wed, 08 Jan 2025 07:31:13 GMT
• Title: Learning Stochastic Nonlinear Dynamics with Embedded Latent Transfer Operators
• Authors: Naichang Ke, Ryogo Tanaka, Yoshinobu Kawahara,
• Abstract summary: We consider an operator-based latent Markov representation of a nonlinear dynamical system. The embedding may be learned simultaneously using reproducing kernels, for example, constructed with feed-forward neural networks. We also address the generalization of sequential state-estimation in nonlinear systems, and of operator-based eigen-mode decomposition of dynamics.
• Score: 5.074981199059028
• License:
• Abstract: We consider an operator-based latent Markov representation of a stochastic nonlinear dynamical system, where the stochastic evolution of the latent state embedded in a reproducing kernel Hilbert space is described with the corresponding transfer operator, and develop a spectral method to learn this representation based on the theory of stochastic realization. The embedding may be learned simultaneously using reproducing kernels, for example, constructed with feed-forward neural networks. We also address the generalization of sequential state-estimation (Kalman filtering) in stochastic nonlinear systems, and of operator-based eigen-mode decomposition of dynamics, for the representation. Several examples with synthetic and real-world data are shown to illustrate the empirical characteristics of our methods, and to investigate the performance of our model in sequential state-estimation and mode decomposition.

Related papers

• On the relationship between Koopman operator approximations and neural ordinary differential equations for data-driven time-evolution predictions [0.0]
  We show that extended dynamic mode decomposition with dictionary learning (EDMD-DL) is equivalent to a neural network representation of the nonlinear discrete-time flow map on the state space. We implement several variations of neural ordinary differential equations (ODEs) and EDMD-DL, developed by combining different aspects of their respective model structures and training procedures. We evaluate these methods using numerical experiments on chaotic dynamics in the Lorenz system and a nine-mode model of turbulent shear flow.
  arXiv  Detail & Related papers  (2024-11-20T00:18:46Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences. We train the model using maximum likelihood estimation with Markov chain Monte Carlo. Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• Sparse identification of quasipotentials via a combined data-driven method [4.599618895656792]
  We leverage on machine learning via the combination of two data-driven techniques, namely a neural network and a sparse regression algorithm, to obtain symbolic expressions of quasipotential functions. We show that our approach discovers a parsimonious quasipotential equation for an archetypal model with a known exact quasipotential and for the dynamics of a nanomechanical resonator.
  arXiv  Detail & Related papers  (2024-07-06T11:27:52Z)
• Discrete, compositional, and symbolic representations through attractor dynamics [51.20712945239422]
  We introduce a novel neural systems model that integrates attractor dynamics with symbolic representations to model cognitive processes akin to the probabilistic language of thought (PLoT) Our model segments the continuous representational space into discrete basins, with attractor states corresponding to symbolic sequences, that reflect the semanticity and compositionality characteristic of symbolic systems through unsupervised learning, rather than relying on pre-defined primitives. This approach establishes a unified framework that integrates both symbolic and sub-symbolic processing through neural dynamics, a neuroplausible substrate with proven expressivity in AI, offering a more comprehensive model that mirrors the complex duality of cognitive operations
  arXiv  Detail & Related papers  (2023-10-03T05:40:56Z)
• Learning minimal representations of stochastic processes with variational autoencoders [52.99137594502433]
  We introduce an unsupervised machine learning approach to determine the minimal set of parameters required to describe a process. Our approach enables for the autonomous discovery of unknown parameters describing processes.
  arXiv  Detail & Related papers  (2023-07-21T14:25:06Z)
• Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems [5.815325960286111]
  We propose a data-driven framework combining Reservoir Computing and Normalizing Flow to study this issue. We verify the effectiveness of the proposed framework in several experiments, including the Van der Pal, El Nino-Southern Oscillation simplified model, and Lorenz system.
  arXiv  Detail & Related papers  (2023-05-01T05:50:17Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Learning and Inference in Sparse Coding Models with Langevin Dynamics [3.0600309122672726]
  We describe a system capable of inference and learning in a probabilistic latent variable model. We demonstrate this idea for a sparse coding model by deriving a continuous-time equation for inferring its latent variables via Langevin dynamics. We show that Langevin dynamics lead to an efficient procedure for sampling from the posterior distribution in the 'L0 sparse' regime, where latent variables are encouraged to be set to zero as opposed to having a small L1 norm.
  arXiv  Detail & Related papers  (2022-04-23T23:16:47Z)
• Structure-Preserving Learning Using Gaussian Processes and Variational Integrators [62.31425348954686]
  We propose the combination of a variational integrator for the nominal dynamics of a mechanical system and learning residual dynamics with Gaussian process regression. We extend our approach to systems with known kinematic constraints and provide formal bounds on the prediction uncertainty.
  arXiv  Detail & Related papers  (2021-12-10T11:09:29Z)
• ImitationFlow: Learning Deep Stable Stochastic Dynamic Systems by Normalizing Flows [29.310742141970394]
  We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, nonlinear dynamics. We show the effectiveness of our method with both standard datasets and a real robot experiment.
  arXiv  Detail & Related papers  (2020-10-25T14:49:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.