On equivalence between linear-chain conditional random fields and hidden Markov chains

• URL: http://arxiv.org/abs/2111.07376v1
• Date: Sun, 14 Nov 2021 15:53:47 GMT
• Title: On equivalence between linear-chain conditional random fields and hidden Markov chains
• Authors: Elie Azeraf, Emmanuel Monfrini, Wojciech Pieczynski
• Abstract summary: Authors usually consider conditional random fields (CRFs) as quite different from generative models. In some areas, like natural language processing (NLP), discriminative models have completely supplanted generative models. We show that HMCs and linear-chain CRFs are not different but just differently parametrized models.
• Score: 6.939768185086753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Practitioners successfully use hidden Markov chains (HMCs) in different problems for about sixty years. HMCs belong to the family of generative models and they are often compared to discriminative models, like conditional random fields (CRFs). Authors usually consider CRFs as quite different from HMCs, and CRFs are often presented as interesting alternative to HMCs. In some areas, like natural language processing (NLP), discriminative models have completely supplanted generative models. However, some recent results show that both families of models are not so different, and both of them can lead to identical processing power. In this paper we compare the simple linear-chain CRFs to the basic HMCs. We show that HMCs are identical to CRFs in that for each CRF we explicitly construct an HMC having the same posterior distribution. Therefore, HMCs and linear-chain CRFs are not different but just differently parametrized models.

Related papers

• Linear chain conditional random fields, hidden Markov models, and related classifiers [4.984601297028258]
  Conditional Random Fields (CRFs) are an alternative to Hidden Markov Models (HMMs) We show that basic Linear-Chain CRFs (LC-CRFs) are in fact equivalent to them in the sense that for each LC-CRF there exists a HMM. We show that it is possible to reformulate the generative Bayesian classifiers Maximum Posterior Mode (MPM) and Maximum a Posteriori (MAP) used in HMMs, as discriminative ones.
  arXiv  Detail & Related papers  (2023-01-03T18:52:39Z)
• Max Markov Chain [4.531240717484252]
  We introduce Max Markov Chain (MMC), a novel representation for a useful subset of High-order Markov Chains (HMCs) MMC is parsimony while retaining the expressiveness of HMCs. We show that MMC is a valuable alternative for modeling processes and has many potential applications.
  arXiv  Detail & Related papers  (2022-11-02T21:50:54Z)
• Expressivity of Hidden Markov Chains vs. Recurrent Neural Networks from a system theoretic viewpoint [0.0]
  We first consider Hidden Markov Chains (HMC) and Recurrent Neural Networks (RNN) as generative models. We embed both structures in a common generative unified model (GUM)
  arXiv  Detail & Related papers  (2022-08-17T09:23:41Z)
• Linear Connectivity Reveals Generalization Strategies [54.947772002394736]
  Some pairs of finetuned models have large barriers of increasing loss on the linear paths between them. We find distinct clusters of models which are linearly connected on the test loss surface, but are disconnected from models outside the cluster. Our work demonstrates how the geometry of the loss surface can guide models towards different functions.
  arXiv  Detail & Related papers  (2022-05-24T23:43:02Z)
• Fuzzy Cognitive Maps and Hidden Markov Models: Comparative Analysis of Efficiency within the Confines of the Time Series Classification Task [0.0]
  We explore the application of Hidden Markov Model (HMM) for time series classification. We identify four models, HMM NN (HMM, one per series), HMM 1C (HMM, one per class), FCM NN, and FCM 1C are then studied in a series of experiments.
  arXiv  Detail & Related papers  (2022-04-28T12:41:05Z)
• Learning Hidden Markov Models When the Locations of Missing Observations are Unknown [54.40592050737724]
  We consider the general problem of learning an HMM from data with unknown missing observation locations. We provide reconstruction algorithms that do not require any assumptions about the structure of the underlying chain. We show that under proper specifications one can reconstruct the process dynamics as well as if the missing observations positions were known.
  arXiv  Detail & Related papers  (2022-03-12T22:40:43Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• Task-agnostic Continual Learning with Hybrid Probabilistic Models [75.01205414507243]
  We propose HCL, a Hybrid generative-discriminative approach to Continual Learning for classification. The flow is used to learn the data distribution, perform classification, identify task changes, and avoid forgetting. We demonstrate the strong performance of HCL on a range of continual learning benchmarks such as split-MNIST, split-CIFAR, and SVHN-MNIST.
  arXiv  Detail & Related papers  (2021-06-24T05:19:26Z)
• What Are Bayesian Neural Network Posteriors Really Like? [63.950151520585024]
  We show that Hamiltonian Monte Carlo can achieve significant performance gains over standard and deep ensembles. We also show that deep distributions are similarly close to HMC as standard SGLD, and closer than standard variational inference.
  arXiv  Detail & Related papers  (2021-04-29T15:38:46Z)
• Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept [56.46135010588918]
  We prove that the widely used class of RNN-Transducer models and segmental models (direct HMM) are equivalent. It is shown that blank probabilities translate into segment length probabilities and vice versa.
  arXiv  Detail & Related papers  (2021-04-13T11:20:48Z)
• Joints in Random Forests [13.096855747795303]
  Decision Trees (DTs) and Random Forests (RFs) are powerful discriminative learners and tools of central importance to the everyday machine learning practitioner and data scientist. We show that DTs and RFs can naturally be interpreted as generative models, by drawing a connection to Probabilistic Circuits. This reinterpretation equips them with a full joint distribution over the feature space and leads to Generative Decision Trees (GeDTs) and Generative Forests (GeFs)
  arXiv  Detail & Related papers  (2020-06-25T14:17:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.