Regular-pattern-sensitive CRFs for Distant Label Interactions

• URL: http://arxiv.org/abs/2411.12484v1
• Date: Tue, 19 Nov 2024 13:08:03 GMT
• Title: Regular-pattern-sensitive CRFs for Distant Label Interactions
• Authors: Sean Papay, Roman Klinger, Sebastian Pado,
• Abstract summary: Regular-pattern-sensitive CRFs (RPCRFs) are a method of enriching standard linear-chain CRFs with the ability to learn long-distance label interactions. We show how a RPCRF can be automatically constructed from a set of user-specified patterns, and demonstrate the model's effectiveness on synthetic data.
• Score: 10.64258723923874
• License:
• Abstract: Linear-chain conditional random fields (CRFs) are a common model component for sequence labeling tasks when modeling the interactions between different labels is important. However, the Markov assumption limits linear-chain CRFs to only directly modeling interactions between adjacent labels. Weighted finite-state transducers (FSTs) are a related approach which can be made to model distant label-label interactions, but exact label inference is intractable for these models in the general case, and the task of selecting an appropriate automaton structure for the desired interaction types poses a practical challenge. In this work, we present regular-pattern-sensitive CRFs (RPCRFs), a method of enriching standard linear-chain CRFs with the ability to learn long-distance label interactions which occur in user-specified patterns. This approach allows users to write regular-expression label patterns concisely specifying which types of interactions the model should take into account, allowing the model to learn from data whether and in which contexts these patterns occur. The result can be interpreted alternatively as a CRF augmented with additional, non-local potentials, or as a finite-state transducer whose structure is defined by a set of easily-interpretable patterns. Critically, unlike the general case for FSTs (and for non-chain CRFs), exact training and inference are tractable for many pattern sets. In this work, we detail how a RPCRF can be automatically constructed from a set of user-specified patterns, and demonstrate the model's effectiveness on synthetic data, showing how different types of patterns can capture different nonlocal dependency structures in label sequences.

Related papers

• FreDF: Learning to Forecast in Frequency Domain [56.24773675942897]
  Time series modeling is uniquely challenged by the presence of autocorrelation in both historical and label sequences. We introduce the Frequency-enhanced Direct Forecast (FreDF) which bypasses the complexity of label autocorrelation by learning to forecast in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-04T08:23:41Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision [75.1860418333995]
  Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently. The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources as labeling functions. Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling the underlying generative process.
  arXiv  Detail & Related papers  (2022-10-06T07:28:53Z)
• Dependency Structure Misspecification in Multi-Source Weak Supervision Models [15.125993628007972]
  We study the effects of label model misspecification on test set performance of a downstream classifier. We derive novel theoretical bounds on the modeling error and empirically show that this error can be substantial.
  arXiv  Detail & Related papers  (2021-06-18T18:15:44Z)
• Constraining Linear-chain CRFs to Regular Languages [10.759863489447204]
  A major challenge in structured prediction is to represent the interdependencies within output structures. We present a generalization of CRFs that can enforce a broad class of constraints, including nonlocal ones. We prove that constrained training is never worse than constrained decoding, and show empirically that it can be substantially better in practice.
  arXiv  Detail & Related papers  (2021-06-14T11:23:59Z)
• Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept [56.46135010588918]
  We prove that the widely used class of RNN-Transducer models and segmental models (direct HMM) are equivalent. It is shown that blank probabilities translate into segment length probabilities and vice versa.
  arXiv  Detail & Related papers  (2021-04-13T11:20:48Z)
• Neural Latent Dependency Model for Sequence Labeling [47.32215014130811]
  A classic approach to sequence labeling is linear chain conditional random fields (CRFs) One limitation of linear chain CRFs is their inability to model long-range dependencies between labels. High order CRFs extend linear chain CRFs by no longer than their order, but the computational complexity grows exponentially in the order. We propose a Neural Latent Dependency Model (NLDM) that models arbitrary length between labels with a latent tree structure.
  arXiv  Detail & Related papers  (2020-11-10T10:05:21Z)
• Robust Question Answering Through Sub-part Alignment [53.94003466761305]
  We model question answering as an alignment problem. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.
  arXiv  Detail & Related papers  (2020-04-30T09:10:57Z)
• Multi-Label Text Classification using Attention-based Graph Neural Network [0.0]
  A graph attention network-based model is proposed to capture the attentive dependency structure among the labels. The proposed model achieves similar or better performance compared to the previous state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-22T17:12:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.