Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows

• URL: http://arxiv.org/abs/2102.07284v1
• Date: Mon, 15 Feb 2021 00:40:30 GMT
• Title: Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows
• Authors: Anubhab Ghosh, Antoine Honor\'e, Dong Liu, Gustav Eje Henter, Saikat Chatterjee
• Abstract summary: We use a generative model that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.
• Score: 25.543231171094384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We test the robustness of a maximum-likelihood (ML) based classifier where sequential data as observation is corrupted by noise. The hypothesis is that a generative model, that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM, can provide a robust classification performance. The combined model is called normalizing-flow mixture model based HMM (NMM-HMM). It can be trained using a combination of expectation-maximization (EM) and backpropagation. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.

Related papers

• Fuzzy Cognitive Maps and Hidden Markov Models: Comparative Analysis of Efficiency within the Confines of the Time Series Classification Task [0.0]
  We explore the application of Hidden Markov Model (HMM) for time series classification. We identify four models, HMM NN (HMM, one per series), HMM 1C (HMM, one per class), FCM NN, and FCM 1C are then studied in a series of experiments.
  arXiv  Detail & Related papers  (2022-04-28T12:41:05Z)
• A new perspective on probabilistic image modeling [92.89846887298852]
  We present a new probabilistic approach for image modeling capable of density estimation, sampling and tractable inference. DCGMMs can be trained end-to-end by SGD from random initial conditions, much like CNNs. We show that DCGMMs compare favorably to several recent PC and SPN models in terms of inference, classification and sampling.
  arXiv  Detail & Related papers  (2022-03-21T14:53:57Z)
• Learning Hidden Markov Models When the Locations of Missing Observations are Unknown [54.40592050737724]
  We consider the general problem of learning an HMM from data with unknown missing observation locations. We provide reconstruction algorithms that do not require any assumptions about the structure of the underlying chain. We show that under proper specifications one can reconstruct the process dynamics as well as if the missing observations positions were known.
  arXiv  Detail & Related papers  (2022-03-12T22:40:43Z)
• Learning Circular Hidden Quantum Markov Models: A Tensor Network Approach [34.77250498401055]
  We show that c-HQMMs are equivalent to a constrained tensor network. This equivalence enables us to provide an efficient learning model for c-HQMMs. The proposed learning approach is evaluated on six real datasets.
  arXiv  Detail & Related papers  (2021-10-29T23:09:31Z)
• Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability [25.543231171094384]
  In pursuit of explainability, we develop generative models for sequential data. We combine modern neural networks (normalizing flows) and traditional generative models (hidden Markov models - HMMs) The proposed generative models can compute likelihood of a data and hence directly suitable for maximum-likelihood (ML) classification approach.
  arXiv  Detail & Related papers  (2021-07-01T20:10:55Z)
• Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning [66.44344616836158]
  We propose an analysis framework that links the pretraining and downstream tasks with an underlying latent variable generative model of text. We show that 1) under certain non-degeneracy conditions on the HMM, simple classification heads can solve the downstream task, 2) prompt tuning obtains downstream guarantees with weaker non-degeneracy conditions, and 3) our recovery guarantees for the memory-augmented HMM are stronger than for the vanilla HMM.
  arXiv  Detail & Related papers  (2021-06-17T03:31:47Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Malware Classification with GMM-HMM Models [8.02151721194722]
  In this paper, we use GMM-HMMs for malware classification and we compare our results to those obtained using discrete HMMs. For our opcode features, GMM-HMMs produce results that are comparable to those obtained using discrete HMMs.
  arXiv  Detail & Related papers  (2021-03-03T23:23:48Z)
• DenseHMM: Learning Hidden Markov Models by Learning Dense Representations [0.0]
  We propose a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.
  arXiv  Detail & Related papers  (2020-12-17T17:48:27Z)
• Scaling Hidden Markov Language Models [118.55908381553056]
  This work revisits the challenge of scaling HMMs to language modeling datasets. We propose methods for scaling HMMs to massive state spaces while maintaining efficient exact inference, a compact parameterization, and effective regularization.
  arXiv  Detail & Related papers  (2020-11-09T18:51:55Z)
• Semi-Supervised Learning with Normalizing Flows [54.376602201489995]
  FlowGMM is an end-to-end approach to generative semi supervised learning with normalizing flows. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data.
  arXiv  Detail & Related papers  (2019-12-30T17:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.