Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer

• URL: http://arxiv.org/abs/2309.07648v2
• Date: Sat, 8 Jun 2024 13:08:39 GMT
• Title: Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer
• Authors: Peng Wang, Yifan Yang, Zheng Liang, Tian Tan, Shiliang Zhang, Xie Chen,
• Abstract summary: We propose C-FNT, a novel E2E model that incorporates class-based LMs into FNT. In C-FNT, the LM score of named entities can be associated with the name class instead of its surface form. The experimental results show that our proposed C-FNT significantly reduces error in named entities without hurting performance in general word recognition.
• Score: 50.572974726351504
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite advancements of end-to-end (E2E) models in speech recognition, named entity recognition (NER) is still challenging but critical for semantic understanding. Previous studies mainly focus on various rule-based or attention-based contextual biasing algorithms. However, their performance might be sensitive to the biasing weight or degraded by excessive attention to the named entity list, along with a risk of false triggering. Inspired by the success of the class-based language model (LM) in NER in conventional hybrid systems and the effective decoupling of acoustic and linguistic information in the factorized neural Transducer (FNT), we propose C-FNT, a novel E2E model that incorporates class-based LMs into FNT. In C-FNT, the LM score of named entities can be associated with the name class instead of its surface form. The experimental results show that our proposed C-FNT significantly reduces error in named entities without hurting performance in general word recognition.

Related papers

• On Significance of Subword tokenization for Low Resource and Efficient Named Entity Recognition: A case study in Marathi [1.6383036433216434]
  We focus on NER for low-resource language and present our case study in the context of the Indian language Marathi. We propose a hybrid approach for efficient NER by integrating a BERT-based subword tokenizer into vanilla CNN/LSTM models. We show that this simple approach of replacing a traditional word-based tokenizer with a BERT-tokenizer brings the accuracy of vanilla single-layer models closer to that of deep pre-trained models like BERT.
  arXiv  Detail & Related papers  (2023-12-03T06:53:53Z)
• Generative error correction for code-switching speech recognition using large language models [49.06203730433107]
  Code-switching (CS) speech refers to the phenomenon of mixing two or more languages within the same sentence. We propose to leverage large language models (LLMs) and lists of hypotheses generated by an ASR to address the CS problem.
  arXiv  Detail & Related papers  (2023-10-17T14:49:48Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Joint Speech Translation and Named Entity Recognition [17.305879157385675]
  A critical task is enriching the output with information regarding the mentioned entities. In this paper we propose multitask models that jointly perform named entity recognition (NER) and entity linking systems. The experimental results show that our models significantly outperform the cascade on the NER task (by 0.4-1.0 F1), without degradation in terms of translation quality.
  arXiv  Detail & Related papers  (2022-10-21T14:24:46Z)
• WCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition [15.446770390648874]
  We propose a novel named entity recognition model WCL-BBCD (Word Contrastive Learning with BERT-BiLSTM-CRF-DBpedia) The model first trains the sentence pairs in the text, calculate similarity between words in sentence pairs by cosine similarity, and fine-tunes the BERT model used for the named entity recognition task through the similarity. Finally, the recognition results are corrected in combination with prior knowledge such as knowledge graphs, so as to alleviate the recognition caused by word abbreviations low-rate problem.
  arXiv  Detail & Related papers  (2022-03-14T08:29:58Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)
• Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference [59.62779187457773]
  We propose a generative classifier for natural language inference (NLI) We compare it to five baselines, including discriminative models and large-scale pretrained language representation models like BERT. Experiments show that GenNLI outperforms both discriminative and pretrained baselines across several challenging NLI experimental settings.
  arXiv  Detail & Related papers  (2020-10-08T04:44:00Z)
• An Evaluation of Recent Neural Sequence Tagging Models in Turkish Named Entity Recognition [5.161531917413708]
  We propose a transformer-based network with a conditional random field layer that leads to the state-of-the-art result. Our study contributes to the literature that quantifies the impact of transfer learning on processing morphologically rich languages.
  arXiv  Detail & Related papers  (2020-05-14T06:54:07Z)
• Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve [49.878051587667244]
  We examine the performance of several variants of LSTM-CRF architectures for named entity recognition. We find that context representations do contribute to system performance, but that the main factor driving high performance is learning the name tokens themselves. We enlist human annotators to evaluate the feasibility of inferring entity types from the context alone and find that, while people are not able to infer the entity type either for the majority of the errors made by the context-only system, there is some room for improvement.
  arXiv  Detail & Related papers  (2020-04-09T14:37:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.