Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows

• URL: http://arxiv.org/abs/2002.10516v4
• Date: Tue, 13 Jul 2021 04:10:23 GMT
• Title: Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows
• Authors: Ruizhi Deng, Bo Chang, Marcus A. Brubaker, Greg Mori, Andreas Lehrmann
• Abstract summary: We propose a novel type of flow driven by a differential deformation of the Wiener process. As a result, we obtain a rich time series model whose observable process inherits many of the appealing properties of its base process.
• Score: 40.9137348900942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Normalizing flows transform a simple base distribution into a complex target distribution and have proved to be powerful models for data generation and density estimation. In this work, we propose a novel type of normalizing flow driven by a differential deformation of the Wiener process. As a result, we obtain a rich time series model whose observable process inherits many of the appealing properties of its base process, such as efficient computation of likelihoods and marginals. Furthermore, our continuous treatment provides a natural framework for irregular time series with an independent arrival process, including straightforward interpolation. We illustrate the desirable properties of the proposed model on popular stochastic processes and demonstrate its superior flexibility to variational RNN and latent ODE baselines in a series of experiments on synthetic and real-world data.

Related papers

• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces a novel family of deep dynamical models designed to represent continuous-time sequence data. We train the model using maximum likelihood estimation with Markov chain Monte Carlo. Experiments on oscillating systems, videos and real-world state sequences (MuJoCo) illustrate that ODEs with the learnable energy-based prior outperform existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Time-changed normalizing flows for accurate SDE modeling [5.402030962296633]
  We propose a novel transformation of dynamic normalizing flows, based on time deformation of a Brownian motion. This approach enables us to effectively model some SDEs, that cannot be modeled otherwise.
  arXiv  Detail & Related papers  (2023-12-22T13:57:29Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Data-driven Modeling and Inference for Bayesian Gaussian Process ODEs via Double Normalizing Flows [28.62579476863723]
  We introduce normalizing flows to re parameterize the ODE vector field, resulting in a data-driven prior distribution. We also apply normalizing flows to the posterior inference of GP ODEs to resolve the issue of strong mean-field assumptions. We validate the effectiveness of our approach on simulated dynamical systems and real-world human motion data.
  arXiv  Detail & Related papers  (2023-09-17T09:28:47Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Normalizing Flow with Variational Latent Representation [20.038183566389794]
  We propose a new framework based on variational latent representation to improve the practical performance of Normalizing Flow (NF) The idea is to replace the standard normal latent variable with a more general latent representation, jointly learned via Variational Bayes. The resulting method is significantly more powerful than the standard normalization flow approach for generating data distributions with multiple modes.
  arXiv  Detail & Related papers  (2022-11-21T16:51:49Z)
• Flow-based Spatio-Temporal Structured Prediction of Motion Dynamics [21.24885597341643]
  Conditional Flows (CNFs) are flexible generative models capable of representing complicated distributions with high dimensionality and interdimensional correlations. We propose MotionFlow as a novel approach that autoregressively normalizes the output on the temporal input features. We apply our method to different tasks, including prediction, motion prediction time series forecasting, and binary segmentation.
  arXiv  Detail & Related papers  (2021-04-09T14:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.