Comparative Analysis of the Hidden Markov Model and LSTM: A Simulative Approach

• URL: http://arxiv.org/abs/2008.03825v1
• Date: Sun, 9 Aug 2020 22:13:10 GMT
• Title: Comparative Analysis of the Hidden Markov Model and LSTM: A Simulative Approach
• Authors: Manie Tadayon, Greg Pottie
• Abstract summary: We show that a hidden Markov model can still be an effective method to process the sequence data even when the first-order Markov assumption is not satisfied. Our results indicate that even an unsupervised hidden Markov model can outperform LSTM when a massive amount of labeled data is not available.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series and sequential data have gained significant attention recently since many real-world processes in various domains such as finance, education, biology, and engineering can be modeled as time series. Although many algorithms and methods such as the Kalman filter, hidden Markov model, and long short term memory (LSTM) are proposed to make inferences and predictions for the data, their usage significantly depends on the application, type of the problem, available data, and sufficient accuracy or loss. In this paper, we compare the supervised and unsupervised hidden Markov model to LSTM in terms of the amount of data needed for training, complexity, and forecasting accuracy. Moreover, we propose various techniques to discretize the observations and convert the problem to a discrete hidden Markov model under stationary and non-stationary situations. Our results indicate that even an unsupervised hidden Markov model can outperform LSTM when a massive amount of labeled data is not available. Furthermore, we show that the hidden Markov model can still be an effective method to process the sequence data even when the first-order Markov assumption is not satisfied.

Related papers

• Dynamical mixture modeling with fast, automatic determination of Markov chains [0.0]
  Variational EM efficiently identifies the number of Markov chains and dynamics of each chain without expensive model comparisons or posterior sampling. The approach is supported by a theoretical analysis and numerical experiments, including simulated and observational data sets based on $tt Last.fm$ music listening, ultramarathon running, and gene expression.
  arXiv  Detail & Related papers  (2024-06-07T05:43:11Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Context-specific kernel-based hidden Markov model for time series analysis [9.007829035130886]
  We introduce a new hidden Markov model based on kernel density estimation. It is capable of capturing kernel dependencies using context-specific Bayesian networks. The benefits in likelihood and classification accuracy from the proposed model are quantified and analyzed.
  arXiv  Detail & Related papers  (2023-01-24T09:10:38Z)
• Learning Hidden Markov Models When the Locations of Missing Observations are Unknown [54.40592050737724]
  We consider the general problem of learning an HMM from data with unknown missing observation locations. We provide reconstruction algorithms that do not require any assumptions about the structure of the underlying chain. We show that under proper specifications one can reconstruct the process dynamics as well as if the missing observations positions were known.
  arXiv  Detail & Related papers  (2022-03-12T22:40:43Z)
• Learning Mixtures of Linear Dynamical Systems [94.49754087817931]
  We develop a two-stage meta-algorithm to efficiently recover each ground-truth LDS model up to error $tildeO(sqrtd/T)$. We validate our theoretical studies with numerical experiments, confirming the efficacy of the proposed algorithm.
  arXiv  Detail & Related papers  (2022-01-26T22:26:01Z)
• Time Series Anomaly Detection with label-free Model Selection [0.6303112417588329]
  We propose LaF-AD, a novel anomaly detection algorithm with label-free model selection for unlabeled times-series data. Our algorithm is easily parallelizable, more robust for ill-conditioned and seasonal data, and highly scalable for a large number of anomaly models.
  arXiv  Detail & Related papers  (2021-06-11T00:21:06Z)
• Markov Modeling of Time-Series Data using Symbolic Analysis [8.522582405896653]
  We will review the different techniques for discretization and memory estimation for discrete processes. We will present some results from literature on partitioning from dynamical systems theory and order estimation using concepts of information theory and statistical learning.
  arXiv  Detail & Related papers  (2021-03-20T20:31:21Z)
• Partially Hidden Markov Chain Linear Autoregressive model: inference and forecasting [0.0]
  Time series subject to change in regime have attracted much interest in domains such as econometry, finance or meteorology. We present a novel model which addresses the intermediate case: (i) state processes associated to such time series are modelled by Partially Hidden Markov Chains (PHMCs) We propose a hidden state inference procedure and a forecasting function that take into account the observed states when existing.
  arXiv  Detail & Related papers  (2021-02-24T22:12:05Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Least Squares Regression with Markovian Data: Fundamental Limits and Algorithms [69.45237691598774]
  We study the problem of least squares linear regression where the data-points are dependent and are sampled from a Markov chain. We establish sharp information theoretic minimax lower bounds for this problem in terms of $tau_mathsfmix$. We propose an algorithm based on experience replay--a popular reinforcement learning technique--that achieves a significantly better error rate.
  arXiv  Detail & Related papers  (2020-06-16T04:26:50Z)
• Targeted stochastic gradient Markov chain Monte Carlo for hidden Markov models with rare latent states [48.705095800341944]
  Markov chain Monte Carlo (MCMC) algorithms for hidden Markov models often rely on the forward-backward sampler. This makes them computationally slow as the length of the time series increases, motivating the development of sub-sampling-based approaches. We propose a targeted sub-sampling approach that over-samples observations corresponding to rare latent states when calculating the gradient of parameters associated with them.
  arXiv  Detail & Related papers  (2018-10-31T17:44:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.