Dynamical mixture modeling with fast, automatic determination of Markov chains

• URL: http://arxiv.org/abs/2406.04653v1
• Date: Fri, 7 Jun 2024 05:43:11 GMT
• Title: Dynamical mixture modeling with fast, automatic determination of Markov chains
• Authors: Christopher E. Miles, Robert J. Webber,
• Abstract summary: Variational EM efficiently identifies the number of Markov chains and dynamics of each chain without expensive model comparisons or posterior sampling. The approach is supported by a theoretical analysis and numerical experiments, including simulated and observational data sets based on $tt Last.fm$ music listening, ultramarathon running, and gene expression.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Markov state modeling has gained popularity in various scientific fields due to its ability to reduce complex time series data into transitions between a few states. Yet, current frameworks are limited by assuming a single Markov chain describes the data, and they suffer an inability to discern heterogeneities. As a solution, this paper proposes a variational expectation-maximization algorithm that identifies a mixture of Markov chains in a time-series data set. The method is agnostic to the definition of the Markov states, whether data-driven (e.g. by spectral clustering) or based on domain knowledge. Variational EM efficiently and organically identifies the number of Markov chains and dynamics of each chain without expensive model comparisons or posterior sampling. The approach is supported by a theoretical analysis and numerical experiments, including simulated and observational data sets based on ${\tt Last.fm}$ music listening, ultramarathon running, and gene expression. The results show the new algorithm is competitive with contemporary mixture modeling approaches and powerful in identifying meaningful heterogeneities in time series data.

Related papers

• From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers [41.82477691012942]
  We study learning a 1-layer self-attention model from a set of prompts and associated output data. We first establish a precise mapping between the self-attention mechanism and Markov models. We characterize an intriguing winner-takes-all phenomenon where the generative process implemented by self-attention collapses into sampling a limited subset of tokens.
  arXiv  Detail & Related papers  (2024-02-21T03:51:34Z)
• Mixture of Coupled HMMs for Robust Modeling of Multivariate Healthcare Time Series [7.5986411724707095]
  We propose a novel class of models, a mixture of coupled hidden Markov models (M-CHMM) To make the model learning feasible, we derive two algorithms to sample the sequences of the latent variables in the CHMM. Compared to existing inference methods, our algorithms are computationally tractable, improve mixing, and allow for likelihood estimation.
  arXiv  Detail & Related papers  (2023-11-14T02:55:37Z)
• Fast and Functional Structured Data Generators Rooted in Out-of-Equilibrium Physics [62.997667081978825]
  We address the challenge of using energy-based models to produce high-quality, label-specific data in structured datasets. Traditional training methods encounter difficulties due to inefficient Markov chain Monte Carlo mixing. We use a novel training algorithm that exploits non-equilibrium effects.
  arXiv  Detail & Related papers  (2023-07-13T15:08:44Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Information Theory Inspired Pattern Analysis for Time-series Data [60.86880787242563]
  We propose a highly generalizable method that uses information theory-based features to identify and learn from patterns in time-series data. For applications with state transitions, features are developed based on Shannon's entropy of Markov chains, entropy rates of Markov chains, and von Neumann entropy of Markov chains. The results show the proposed information theory-based features improve the recall rate, F1 score, and accuracy on average by up to 23.01% compared with the baseline models.
  arXiv  Detail & Related papers  (2023-02-22T21:09:35Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Autoencoder Based Iterative Modeling and Multivariate Time-Series Subsequence Clustering Algorithm [0.0]
  This paper introduces an algorithm for the detection of change-points and the identification of the corresponding subsequences in transient time-series data (MTSD) We use a recurrent neural network (RNN) based Autoencoder (AE) which is iteratively trained on incoming data. A model of the identified subsequence is saved and used for recognition of repeating subsequences as well as fast offline clustering.
  arXiv  Detail & Related papers  (2022-09-09T09:59:56Z)
• Markov Chain Monte Carlo for Continuous-Time Switching Dynamical Systems [26.744964200606784]
  We propose a novel inference algorithm utilizing a Markov Chain Monte Carlo approach. The presented Gibbs sampler allows to efficiently obtain samples from the exact continuous-time posterior processes.
  arXiv  Detail & Related papers  (2022-05-18T09:03:00Z)
• Novel Features for Time Series Analysis: A Complex Networks Approach [62.997667081978825]
  Time series data are ubiquitous in several domains as climate, economics and health care. Recent conceptual approach relies on time series mapping to complex networks. Network analysis can be used to characterize different types of time series.
  arXiv  Detail & Related papers  (2021-10-11T13:46:28Z)
• Comparative Analysis of the Hidden Markov Model and LSTM: A Simulative Approach [0.0]
  We show that a hidden Markov model can still be an effective method to process the sequence data even when the first-order Markov assumption is not satisfied. Our results indicate that even an unsupervised hidden Markov model can outperform LSTM when a massive amount of labeled data is not available.
  arXiv  Detail & Related papers  (2020-08-09T22:13:10Z)
• XEM: An Explainable-by-Design Ensemble Method for Multivariate Time Series Classification [61.33695273474151]
  We present XEM, an eXplainable-by-design Ensemble method for Multivariable time series classification. XEM relies on a new hybrid ensemble method that combines an explicit boosting-bagging approach and an implicit divide-and-conquer approach. Our evaluation shows that XEM outperforms the state-of-the-art MTS classifiers on the public UEA datasets.
  arXiv  Detail & Related papers  (2020-05-07T17:50:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.