Global Attention for Name Tagging

• URL: http://arxiv.org/abs/2010.09270v1
• Date: Mon, 19 Oct 2020 07:27:15 GMT
• Title: Global Attention for Name Tagging
• Authors: Boliang Zhang, Spencer Whitehead, Lifu Huang and Heng Ji
• Abstract summary: We present a new framework to improve name tagging by utilizing local, document-level, and corpus-level contextual information. We propose a model that learns to incorporate document-level and corpus-level contextual information alongside local contextual information via global attentions. Experiments on benchmark datasets show the effectiveness of our approach.
• Score: 56.62059996864408
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many name tagging approaches use local contextual information with much success, but fail when the local context is ambiguous or limited. We present a new framework to improve name tagging by utilizing local, document-level, and corpus-level contextual information. We retrieve document-level context from other sentences within the same document and corpus-level context from sentences in other topically related documents. We propose a model that learns to incorporate document-level and corpus-level contextual information alongside local contextual information via global attentions, which dynamically weight their respective contextual information, and gating mechanisms, which determine the influence of this information. Extensive experiments on benchmark datasets show the effectiveness of our approach, which achieves state-of-the-art results for Dutch, German, and Spanish on the CoNLL-2002 and CoNLL-2003 datasets.

Related papers

• Bridging Local Details and Global Context in Text-Attributed Graphs [62.522550655068336]
  GraphBridge is a framework that bridges local and global perspectives by leveraging contextual textual information. Our method achieves state-of-theart performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues.
  arXiv  Detail & Related papers  (2024-06-18T13:35:25Z)
• The Role of Global and Local Context in Named Entity Recognition [3.1638713158723686]
  This article explores the impact of global document context, and its relationships with local context. We find that correctly retrieving global document context has a greater impact on performance than only leveraging local context.
  arXiv  Detail & Related papers  (2023-05-04T20:22:18Z)
• Named entity recognition architecture combining contextual and global features [5.92351086183376]
  Named entity recognition (NER) is an information extraction technique that aims to locate and classify named entities. We propose the combination of contextual features from XLNet and global features from Graph Convolution Network (GCN) to enhance NER performance.
  arXiv  Detail & Related papers  (2021-12-15T10:54:36Z)
• Exploiting Global Contextual Information for Document-level Named Entity Recognition [46.99922251839363]
  We propose a model called Global Context enhanced Document-level NER (GCDoc) At word-level, a document graph is constructed to model a wider range of dependencies between words. At sentence-level, for appropriately modeling wider context beyond single sentence, we employ a cross-sentence module. Our model reaches F1 score of 92.22 (93.40 with BERT) on CoNLL 2003 dataset and 88.32 (90.49 with BERT) on Ontonotes 5.0 dataset.
  arXiv  Detail & Related papers  (2021-06-02T01:52:07Z)
• Global-to-Local Neural Networks for Document-Level Relation Extraction [11.900280120655898]
  Relation extraction (RE) aims to identify the semantic relations between named entities in text. Recent years have witnessed it raised to the document level, which requires complex reasoning with entities and mentions throughout an entire document. We propose a novel model to document-level RE, by encoding the document information in terms of entity global and local representations as well as context relation representations.
  arXiv  Detail & Related papers  (2020-09-22T07:30:19Z)
• Document-level Neural Machine Translation with Document Embeddings [82.4684444847092]
  This work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings. The proposed document-aware NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end.
  arXiv  Detail & Related papers  (2020-09-16T19:43:29Z)
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation [50.01708049531156]
  We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
  arXiv  Detail & Related papers  (2020-02-24T12:52:10Z)
• Towards Making the Most of Context in Neural Machine Translation [112.9845226123306]
  We argue that previous research did not make a clear use of the global context. We propose a new document-level NMT framework that deliberately models the local context of each sentence.
  arXiv  Detail & Related papers  (2020-02-19T03:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.