Masked Conditional Random Fields for Sequence Labeling

• URL: http://arxiv.org/abs/2103.10682v1
• Date: Fri, 19 Mar 2021 08:23:24 GMT
• Title: Masked Conditional Random Fields for Sequence Labeling
• Authors: Tianwen Wei, Jianwei Qi, Shenghuan He, Songtao Sun
• Abstract summary: Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. We propose Masked Conditional Random Field (MCRF), an easy to implement variant of CRF that impose restrictions on candidate paths during both training and decoding phases. We show that the proposed method thoroughly resolves this issue and brings consistent improvement over existing CRF-based models with near zero additional cost.
• Score: 2.982218441172364
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. Despite its great success, CRF has the shortcoming of occasionally generating illegal sequences of tags, e.g. sequences containing an "I-" tag immediately after an "O" tag, which is forbidden by the underlying BIO tagging scheme. In this work, we propose Masked Conditional Random Field (MCRF), an easy to implement variant of CRF that impose restrictions on candidate paths during both training and decoding phases. We show that the proposed method thoroughly resolves this issue and brings consistent improvement over existing CRF-based models with near zero additional cost.

Related papers

• Bregman Conditional Random Fields: Sequence Labeling with Parallelizable Inference Algorithms [7.690409460019577]
  We propose a novel discriminative model for sequence labeling called Bregman conditional random fields (BCRF)<n>BCRF allows fast parallelizable inference algorithms based on iterative Bregman projections.<n>We show how such models can be learned using Fenchel-Young losses, including extension for learning from partial labels.
  arXiv  Detail & Related papers  (2025-05-31T22:36:21Z)
• Empirical Bound Information-Directed Sampling for Norm-Agnostic Bandits [0.0]
  We introduce a novel frequentist IDS algorithm that iteratively refines a high-probability upper bound on the true parameter norm using accumulating data. We establish regret bounds for our algorithm that do not depend on an initially assumed parameter norm bound and demonstrate that our method outperforms state-of-the-art IDS and UCB algorithms.
  arXiv  Detail & Related papers  (2025-03-07T02:33:37Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• Filtered Semi-Markov CRF [3.839857803092043]
  Semi-Markov CRF has been proposed as an alternative to the traditional Linear Chain CRF for text segmentation tasks. Semi-CRF suffers from two major drawbacks: (1) quadratic complexity over sequence length, and (2) inferior performance compared to CRF for sequence labeling tasks.
  arXiv  Detail & Related papers  (2023-11-29T19:11:55Z)
• Exploiting Low-confidence Pseudo-labels for Source-free Object Detection [54.98300313452037]
  Source-free object detection (SFOD) aims to adapt a source-trained detector to an unlabeled target domain without access to the labeled source data. Current SFOD methods utilize a threshold-based pseudo-label approach in the adaptation phase. We propose a new approach to take full advantage of pseudo-labels by introducing high and low confidence thresholds.
  arXiv  Detail & Related papers  (2023-10-19T12:59:55Z)
• One-shot Generative Distribution Matching for Augmented RF-based UAV Identification [0.0]
  This work addresses the challenge of identifying Unmanned Aerial Vehicles (UAV) using radiofrequency (RF) fingerprinting in limited RF environments. The complexity and variability of RF signals, influenced by environmental interference and hardware imperfections, often render traditional RF-based identification methods ineffective. One-shot generative methods for augmenting transformed RF signals offer a significant improvement in UAV identification.
  arXiv  Detail & Related papers  (2023-01-20T02:35:43Z)
• Uncertainty-Aware Label Refinement for Sequence Labeling [47.67853514765981]
  We introduce a novel two-stage label decoding framework to model long-term label dependencies. A base model first predicts draft labels, and then a novel two-stream self-attention model makes refinements on these draft predictions.
  arXiv  Detail & Related papers  (2020-12-19T06:56:59Z)
• Semi-Supervised Speech Recognition via Graph-based Temporal Classification [59.58318952000571]
  Semi-supervised learning has demonstrated promising results in automatic speech recognition by self-training. The effectiveness of this approach largely relies on the pseudo-label accuracy. Alternative ASR hypotheses of an N-best list can provide more accurate labels for an unlabeled speech utterance.
  arXiv  Detail & Related papers  (2020-10-29T14:56:56Z)
• Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers [15.279850826041066]
  Current work eschews prior knowledge of how the span encoding scheme works and relies on the conditional random field (CRF) learning which transitions are illegal and which are not to facilitate global coherence. We find that by constraining the output to suppress illegal transitions we can train a tagger with a cross-entropy loss twice as fast as a CRF with differences in F1 that are statistically insignificant.
  arXiv  Detail & Related papers  (2020-10-09T04:07:52Z)
• Conditional Hybrid GAN for Sequence Generation [56.67961004064029]
  We propose a novel conditional hybrid GAN (C-Hybrid-GAN) to solve this issue. We exploit the Gumbel-Softmax technique to approximate the distribution of discrete-valued sequences. We demonstrate that the proposed C-Hybrid-GAN outperforms the existing methods in context-conditioned discrete-valued sequence generation.
  arXiv  Detail & Related papers  (2020-09-18T03:52:55Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)
• Speaker-change Aware CRF for Dialogue Act Classification [0.0]
  Recent work in Dialogue Act (DA) classification approaches the task as a sequence labeling problem. This paper proposes a simple modification of the CRF layer that takes speaker-change into account.
  arXiv  Detail & Related papers  (2020-04-06T18:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.