Hidden Markov Chains, Entropic Forward-Backward, and Part-Of-Speech Tagging

• URL: http://arxiv.org/abs/2005.10629v1
• Date: Thu, 21 May 2020 13:31:11 GMT
• Title: Hidden Markov Chains, Entropic Forward-Backward, and Part-Of-Speech Tagging
• Authors: Elie Azeraf, Emmanuel Monfrini, Emmanuel Vignon, Wojciech Pieczynski
• Abstract summary: Hidden Markov Chain (HMC) model associated with classic Forward-Backward probabilities cannot handle arbitrary features. We show that the problem is not due to HMC itself, but to the way its restoration algorithms are computed. We present a new way of computing HMC based restorations using original Entropic Forward and Entropic Backward (EFB) probabilities.
• Score: 5.778730972088575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to take into account the characteristics - also called features - of observations is essential in Natural Language Processing (NLP) problems. Hidden Markov Chain (HMC) model associated with classic Forward-Backward probabilities cannot handle arbitrary features like prefixes or suffixes of any size, except with an independence condition. For twenty years, this default has encouraged the development of other sequential models, starting with the Maximum Entropy Markov Model (MEMM), which elegantly integrates arbitrary features. More generally, it led to neglect HMC for NLP. In this paper, we show that the problem is not due to HMC itself, but to the way its restoration algorithms are computed. We present a new way of computing HMC based restorations using original Entropic Forward and Entropic Backward (EFB) probabilities. Our method allows taking into account features in the HMC framework in the same way as in the MEMM framework. We illustrate the efficiency of HMC using EFB in Part-Of-Speech Tagging, showing its superiority over MEMM based restoration. We also specify, as a perspective, how HMCs with EFB might appear as an alternative to Recurrent Neural Networks to treat sequential data with a deep architecture.

Related papers

• CWF: Consolidating Weak Features in High-quality Mesh Simplification [50.634070540791555]
  We propose a smooth functional that simultaneously considers all of these requirements. The functional comprises a normal anisotropy term and a Centroidal Voronoi Tessellation (CVT) energy term.
  arXiv  Detail & Related papers  (2024-04-24T05:37:17Z)
• Max Markov Chain [4.531240717484252]
  We introduce Max Markov Chain (MMC), a novel representation for a useful subset of High-order Markov Chains (HMCs) MMC is parsimony while retaining the expressiveness of HMCs. We show that MMC is a valuable alternative for modeling processes and has many potential applications.
  arXiv  Detail & Related papers  (2022-11-02T21:50:54Z)
• PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images [60.33197938330409]
  PyMAF-X is a regression-based approach to recovering parametric full-body models from monocular images. PyMAF and PyMAF-X effectively improve the mesh-image alignment and achieve new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-07-13T17:58:33Z)
• Learning Hidden Markov Models When the Locations of Missing Observations are Unknown [54.40592050737724]
  We consider the general problem of learning an HMM from data with unknown missing observation locations. We provide reconstruction algorithms that do not require any assumptions about the structure of the underlying chain. We show that under proper specifications one can reconstruct the process dynamics as well as if the missing observations positions were known.
  arXiv  Detail & Related papers  (2022-03-12T22:40:43Z)
• Efficient Learning and Decoding of the Continuous-Time Hidden Markov Model for Disease Progression Modeling [119.50438407358862]
  We present the first complete characterization of efficient EM-based learning methods for CT-HMM models. We show that EM-based learning consists of two challenges: the estimation of posterior state probabilities and the computation of end-state conditioned statistics. We demonstrate the use of CT-HMMs with more than 100 states to visualize and predict disease progression using a glaucoma dataset and an Alzheimer's disease dataset.
  arXiv  Detail & Related papers  (2021-10-26T20:06:05Z)
• BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition [57.2201011783393]
  conditional hidden Markov model (CHMM) CHMM predicts token-wise transition and emission probabilities from the BERT embeddings of the input tokens. It fine-tunes a BERT-based NER model with the labels inferred by CHMM.
  arXiv  Detail & Related papers  (2021-05-26T21:18:48Z)
• Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows [25.543231171094384]
  We use a generative model that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.
  arXiv  Detail & Related papers  (2021-02-15T00:40:30Z)
• DenseHMM: Learning Hidden Markov Models by Learning Dense Representations [0.0]
  We propose a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.
  arXiv  Detail & Related papers  (2020-12-17T17:48:27Z)
• Scaling Hidden Markov Language Models [118.55908381553056]
  This work revisits the challenge of scaling HMMs to language modeling datasets. We propose methods for scaling HMMs to massive state spaces while maintaining efficient exact inference, a compact parameterization, and effective regularization.
  arXiv  Detail & Related papers  (2020-11-09T18:51:55Z)
• Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting [6.684193501969829]
  Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo approach that exhibits favourable exploration properties in high-dimensional models such as neural networks. We introduce a new integration scheme for split HMC that does not rely on symmetric gradients. Our approach demonstrates HMC as a feasible option when considering inference schemes for large-scale machine learning problems.
  arXiv  Detail & Related papers  (2020-10-14T01:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.