Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition

• URL: http://arxiv.org/abs/2303.10870v1
• Date: Mon, 20 Mar 2023 05:11:22 GMT
• Title: Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition
• Authors: Ying Mo, Hongyin Tang, Jiahao Liu, Qifan Wang, Zenglin Xu, Jingang Wang, Wei Wu, Zhoujun Li
• Abstract summary: Named entity recognition (NER) is an important research problem in natural language processing. This paper proposes a multi-task Transformer, which incorporates an entity boundary detection task into the named entity recognition task.
• Score: 35.44123819012004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Named entity recognition (NER) is an important research problem in natural language processing. There are three types of NER tasks, including flat, nested and discontinuous entity recognition. Most previous sequential labeling models are task-specific, while recent years have witnessed the rising of generative models due to the advantage of unifying all NER tasks into the seq2seq model framework. Although achieving promising performance, our pilot studies demonstrate that existing generative models are ineffective at detecting entity boundaries and estimating entity types. This paper proposes a multi-task Transformer, which incorporates an entity boundary detection task into the named entity recognition task. More concretely, we achieve entity boundary detection by classifying the relations between tokens within the sentence. To improve the accuracy of entity-type mapping during decoding, we adopt an external knowledge base to calculate the prior entity-type distributions and then incorporate the information into the model via the self and cross-attention mechanisms. We perform experiments on an extensive set of NER benchmarks, including two flat, three nested, and three discontinuous NER datasets. Experimental results show that our approach considerably improves the generative NER model's performance.

Related papers

• Leveraging large language models for efficient representation learning for entity resolution [2.035773073690598]
  TriBERTa is a supervised entity resolution system that learns representations for entity matching. The proposed approach outperforms state-of-the-art representations by a margin of 3 - 19%.
  arXiv  Detail & Related papers  (2024-11-15T23:24:07Z)
• ToNER: Type-oriented Named Entity Recognition with Generative Language Model [14.11486479935094]
  We propose a novel NER framework, namely ToNER based on a generative model. In ToNER, a type matching model is proposed at first to identify the entity types most likely to appear in the sentence. We append a multiple binary classification task to fine-tune the generative model's encoder, so as to generate the refined representation of the input sentence.
  arXiv  Detail & Related papers  (2024-04-14T05:13:37Z)
• Hybrid Multi-stage Decoding for Few-shot NER with Entity-aware Contrastive Learning [32.62763647036567]
  Few-shot named entity recognition can identify new types of named entities based on a few labeled examples. We propose the Hybrid Multi-stage Decoding for Few-shot NER with Entity-aware Contrastive Learning (MsFNER) MsFNER splits the general NER into two stages: entity-span detection and entity classification.
  arXiv  Detail & Related papers  (2024-04-10T12:31:09Z)
• In-Context Learning for Few-Shot Nested Named Entity Recognition [53.55310639969833]
  We introduce an effective and innovative ICL framework for the setting of few-shot nested NER. We improve the ICL prompt by devising a novel example demonstration selection mechanism, EnDe retriever. In EnDe retriever, we employ contrastive learning to perform three types of representation learning, in terms of semantic similarity, boundary similarity, and label similarity.
  arXiv  Detail & Related papers  (2024-02-02T06:57:53Z)
• NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval [49.827932299460514]
  We argue that capabilities provided by large language models are not the end of NER research, but rather an exciting beginning. We present three variants of the NER task, together with a dataset to support them. We provide a large, silver-annotated corpus of 4 million paragraphs covering 500 entity types.
  arXiv  Detail & Related papers  (2023-10-22T12:23:00Z)
• Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach [50.12455129619845]
  Named Entity Recognition (NER) aims to extract and classify entity mentions in the text into pre-defined types. We propose to incorporate the label dependencies among entity types into a multi-task learning framework for better MRC-based NER.
  arXiv  Detail & Related papers  (2023-09-20T03:15:05Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• Gaussian Prior Reinforcement Learning for Nested Named Entity Recognition [52.46740830977898]
  We propose a novel seq2seq model named GPRL, which formulates the nested NER task as an entity triplet sequence generation process. Experiments on three nested NER datasets demonstrate that GPRL outperforms previous nested NER models.
  arXiv  Detail & Related papers  (2023-05-12T05:55:34Z)
• Unified Named Entity Recognition as Word-Word Relation Classification [25.801945832005504]
  We present a novel alternative by modeling the unified NER as word-word relation classification, namely W2NER. The architecture resolves the kernel bottleneck of unified NER by effectively modeling the neighboring relations between entity words. Based on the W2NER scheme we develop a neural framework, in which the unified NER is modeled as a 2D grid of word pairs.
  arXiv  Detail & Related papers  (2021-12-19T06:11:07Z)
• AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction [54.20039200180071]
  We present a novel framework to improve NER performance by automatically generating and leveraging entity triggers'' Our framework leverages post-hoc explanation to generate rationales and strengthens a model's prior knowledge using an embedding technique. AutoTriggER shows strong label-efficiency, is capable of generalizing to unseen entities, and outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on average.
  arXiv  Detail & Related papers  (2021-09-10T08:11:56Z)
• A Sequence-to-Set Network for Nested Named Entity Recognition [38.05786148160635]
  We propose a novel sequence-to-set neural network for nested NER. We use a non-autoregressive decoder to predict the final set of entities in one pass. Experimental results show that our proposed model achieves state-of-the-art on three nested NER corpora.
  arXiv  Detail & Related papers  (2021-05-19T03:10:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.