Dynamical Wasserstein Barycenters for Time-series Modeling

• URL: http://arxiv.org/abs/2110.06741v1
• Date: Wed, 13 Oct 2021 14:20:06 GMT
• Title: Dynamical Wasserstein Barycenters for Time-series Modeling
• Authors: Kevin C. Cheng, Shuchin Aeron, Michael C. Hughes, Eric L. Miller
• Abstract summary: Most prior work assumes instantaneous transitions between pure discrete states. We propose a Wasserstein barycentric (DWB) model that estimates the system state over time. Experiments on several human activity datasets show that our proposed DWB model accurately learns the generating distribution of pure states.
• Score: 16.212262513825717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many time series can be modeled as a sequence of segments representing high-level discrete states, such as running and walking in a human activity application. Flexible models should describe the system state and observations in stationary ``pure-state'' periods as well as transition periods between adjacent segments, such as a gradual slowdown between running and walking. However, most prior work assumes instantaneous transitions between pure discrete states. We propose a dynamical Wasserstein barycentric (DWB) model that estimates the system state over time as well as the data-generating distributions of pure states in an unsupervised manner. Our model assumes each pure state generates data from a multivariate normal distribution, and characterizes transitions between states via displacement-interpolation specified by the Wasserstein barycenter. The system state is represented by a barycentric weight vector which evolves over time via a random walk on the simplex. Parameter learning leverages the natural Riemannian geometry of Gaussian distributions under the Wasserstein distance, which leads to improved convergence speeds. Experiments on several human activity datasets show that our proposed DWB model accurately learns the generating distribution of pure states while improving state estimation for transition periods compared to the commonly used linear interpolation mixture models.

Related papers

• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework. We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points. Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces a novel family of deep dynamical models designed to represent continuous-time sequence data. We train the model using maximum likelihood estimation with Markov chain Monte Carlo. Experiments on oscillating systems, videos and real-world state sequences (MuJoCo) illustrate that ODEs with the learnable energy-based prior outperform existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Generative modeling for time series via Schr{\"o}dinger bridge [0.0]
  We propose a novel generative model for time series based on Schr"dinger bridge (SB) approach. This consists in the entropic via optimal transport between a reference probability measure on path space and a target measure consistent with the joint data distribution of the time series.
  arXiv  Detail & Related papers  (2023-04-11T09:45:06Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Nonparametric and Regularized Dynamical Wasserstein Barycenters for Sequential Observations [16.05839190247062]
  We consider probabilistic models for sequential observations which exhibit gradual transitions among a finite number of states. We numerically solve a finite dimensional estimation problem using cyclic descent alternating between updates to the pure-state quantile functions and the barycentric weights. We demonstrate the utility of the proposed algorithm in segmenting both simulated and real world human activity time series.
  arXiv  Detail & Related papers  (2022-10-04T21:39:55Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion [88.45326906116165]
  We present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID) We encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories. Experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-03-25T16:59:08Z)
• Online Time Series Anomaly Detection with State Space Gaussian Processes [12.483273106706623]
  R-ssGPFA is an unsupervised online anomaly detection model for uni- and multivariate time series. For high-dimensional time series, we propose an extension of Gaussian process factor analysis to identify the common latent processes of the time series. Our model's robustness is improved by using a simple to skip Kalman updates when encountering anomalous observations.
  arXiv  Detail & Related papers  (2022-01-18T06:43:32Z)
• Traversing Time with Multi-Resolution Gaussian Process State-Space Models [17.42262122708566]
  We propose a novel Gaussian process state-space architecture composed of multiple components, each trained on a different resolution, to model effects on different timescales. We benchmark our novel method on semi-synthetic data and on an engine modeling task. In both experiments, our approach compares favorably against its state-of-the-art alternatives that operate on a single time-scale only.
  arXiv  Detail & Related papers  (2021-12-06T18:39:27Z)
• Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows [40.9137348900942]
  We propose a novel type of flow driven by a differential deformation of the Wiener process. As a result, we obtain a rich time series model whose observable process inherits many of the appealing properties of its base process.
  arXiv  Detail & Related papers  (2020-02-24T20:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.