Filtered Semi-Markov CRF

• URL: http://arxiv.org/abs/2311.18028v1
• Date: Wed, 29 Nov 2023 19:11:55 GMT
• Title: Filtered Semi-Markov CRF
• Authors: Urchade Zaratiana, Nadi Tomeh, Niama El Khbir, Pierre Holat, Thierry Charnois
• Abstract summary: Semi-Markov CRF has been proposed as an alternative to the traditional Linear Chain CRF for text segmentation tasks. Semi-CRF suffers from two major drawbacks: (1) quadratic complexity over sequence length, and (2) inferior performance compared to CRF for sequence labeling tasks.
• Score: 3.839857803092043
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-Markov CRF has been proposed as an alternative to the traditional Linear Chain CRF for text segmentation tasks such as Named Entity Recognition (NER). Unlike CRF, which treats text segmentation as token-level prediction, Semi-CRF considers segments as the basic unit, making it more expressive. However, Semi-CRF suffers from two major drawbacks: (1) quadratic complexity over sequence length, as it operates on every span of the input sequence, and (2) inferior performance compared to CRF for sequence labeling tasks like NER. In this paper, we introduce Filtered Semi-Markov CRF, a variant of Semi-CRF that addresses these issues by incorporating a filtering step to eliminate irrelevant segments, reducing complexity and search space. Our approach is evaluated on several NER benchmarks, where it outperforms both CRF and Semi-CRF while being significantly faster. The implementation of our method is available on \href{https://github.com/urchade/Filtered-Semi-Markov-CRF}{Github}.

Related papers

• Pay Attention to CTC: Fast and Robust Pseudo-Labelling for Unified Speech Recognition [61.39209522608919]
  Unified Speech Recognition has emerged as a semi-supervised framework for training a single model for audio, visual, and audiovisual speech recognition.<n>We propose CTC-driven teacher forcing, where greedily decoded CTC pseudo-labels are fed into the decoder to generate attention targets.<n>Because CTC and CTC-driven attention pseudo-labels have the same length, the decoder can predict both simultaneously.
  arXiv  Detail & Related papers  (2026-02-22T19:38:21Z)
• SANeRF-HQ: Segment Anything for NeRF in High Quality [61.77762568224097]
  We introduce the Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high-quality 3D segmentation of any target object in a given scene. We employ density field and RGB similarity to enhance the accuracy of segmentation boundary during the aggregation.
  arXiv  Detail & Related papers  (2023-12-03T23:09:38Z)
• Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition [77.4863142882136]
  Cross-encoder models are prohibitively expensive for direct k-nearest neighbor (k-NN) search. We propose ADACUR, a method that adaptively, iteratively, and efficiently minimizes the approximation error for the practically important top-k neighbors.
  arXiv  Detail & Related papers  (2023-05-04T17:01:17Z)
• Retinal Vessel Segmentation with Pixel-wise Adaptive Filters [47.8629995041574]
  We propose two novel methods to address the challenges of retinal vessel segmentation. First, we devise a light-weight module, named multi-scale residual similarity gathering (MRSG), to generate pixel-wise adaptive filters (PA-Filters) Second, we introduce a response cue erasing (RCE) strategy to enhance the segmentation accuracy.
  arXiv  Detail & Related papers  (2022-02-03T14:40:36Z)
• ES-CRF: Embedded Superpixel CRF for Semantic Segmentation [9.759391777814619]
  We propose a novel method named Embedded Superpixel CRF (ES-CRF) to purify the feature representation of boundary pixels. ES-CRF fuses the CRF mechanism into the CNN network as an organic whole for more effective end-to-end optimization. It yields new records on two challenging benchmarks, i.e., Cityscapes and ADE20K.
  arXiv  Detail & Related papers  (2021-12-14T02:06:28Z)
• Sequence Transduction with Graph-based Supervision [96.04967815520193]
  We present a new transducer objective function that generalizes the RNN-T loss to accept a graph representation of the labels. We demonstrate that transducer-based ASR with CTC-like lattice achieves better results compared to standard RNN-T.
  arXiv  Detail & Related papers  (2021-11-01T21:51:42Z)
• Constraining Linear-chain CRFs to Regular Languages [10.759863489447204]
  A major challenge in structured prediction is to represent the interdependencies within output structures. We present a generalization of CRFs that can enforce a broad class of constraints, including nonlocal ones. We prove that constrained training is never worse than constrained decoding, and show empirically that it can be substantially better in practice.
  arXiv  Detail & Related papers  (2021-06-14T11:23:59Z)
• Masked Conditional Random Fields for Sequence Labeling [2.982218441172364]
  Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. We propose Masked Conditional Random Field (MCRF), an easy to implement variant of CRF that impose restrictions on candidate paths during both training and decoding phases. We show that the proposed method thoroughly resolves this issue and brings consistent improvement over existing CRF-based models with near zero additional cost.
  arXiv  Detail & Related papers  (2021-03-19T08:23:24Z)
• Efficient semidefinite-programming-based inference for binary and multi-class MRFs [83.09715052229782]
  We propose an efficient method for computing the partition function or MAP estimate in a pairwise MRF. We extend semidefinite relaxations from the typical binary MRF to the full multi-class setting, and develop a compact semidefinite relaxation that can again be solved efficiently using the solver.
  arXiv  Detail & Related papers  (2020-12-04T15:36:29Z)
• Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers [15.279850826041066]
  Current work eschews prior knowledge of how the span encoding scheme works and relies on the conditional random field (CRF) learning which transitions are illegal and which are not to facilitate global coherence. We find that by constraining the output to suppress illegal transitions we can train a tagger with a cross-entropy loss twice as fast as a CRF with differences in F1 that are statistically insignificant.
  arXiv  Detail & Related papers  (2020-10-09T04:07:52Z)
• AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network [75.44925576268052]
  The linear-chain Conditional Random Field (CRF) model is one of the most widely-used neural sequence labeling approaches. Exact probabilistic inference algorithms are typically applied in training and prediction stages of the CRF model. We propose to employ a parallelizable approximate variational inference algorithm for the CRF model.
  arXiv  Detail & Related papers  (2020-09-17T12:18:43Z)
• Fast and Accurate Neural CRF Constituency Parsing [16.90190521285297]
  This work presents a fast and accurate neural CRF constituency computation. We batchify the inside algorithm for loss by direct large tensor operations on GPU, and avoid the outside algorithm for computation via efficient back-propagation. Experiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF achieves new state-of-the-art performance on both settings of w/o and w/ BERT.
  arXiv  Detail & Related papers  (2020-08-09T14:38:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.