The Role of Global and Local Context in Named Entity Recognition

• URL: http://arxiv.org/abs/2305.03132v2
• Date: Tue, 30 May 2023 21:26:33 GMT
• Title: The Role of Global and Local Context in Named Entity Recognition
• Authors: Arthur Amalvy, Vincent Labatut, Richard Dufour
• Abstract summary: This article explores the impact of global document context, and its relationships with local context. We find that correctly retrieving global document context has a greater impact on performance than only leveraging local context.
• Score: 3.1638713158723686
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Pre-trained transformer-based models have recently shown great performance when applied to Named Entity Recognition (NER). As the complexity of their self-attention mechanism prevents them from processing long documents at once, these models are usually applied in a sequential fashion. Such an approach unfortunately only incorporates local context and prevents leveraging global document context in long documents such as novels, which might hinder performance. In this article, we explore the impact of global document context, and its relationships with local context. We find that correctly retrieving global document context has a greater impact on performance than only leveraging local context, prompting for further research on how to better retrieve that context.

Related papers

• Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset [6.633914491587503]
  We propose to generate a synthetic context retrieval training dataset using Alpaca. Using this dataset, we train a neural context retriever based on a BERT model that is able to find relevant context for NER. We show that our method outperforms several retrieval baselines for the NER task on an English literary dataset composed of the first chapter of 40 books.
  arXiv  Detail & Related papers  (2023-10-16T06:53:12Z)
• CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion [68.19934563919192]
  We propose a curriculum sampling strategy that utilizes pseudo queries during training and progressively enhances the relevance between the generated query and the real query. Experimental results on both in-domain and out-of-domain datasets demonstrate that our approach outperforms previous dense retrieval models.
  arXiv  Detail & Related papers  (2022-12-18T15:57:46Z)
• Dynamic Global Memory for Document-level Argument Extraction [63.314514124716936]
  We introduce a new global neural generation-based framework for document-level event argument extraction. We use a document memory store to record the contextual event information and leverage it to implicitly and explicitly help with decoding of arguments for later events. Empirical results show that our framework outperforms prior methods substantially.
  arXiv  Detail & Related papers  (2022-09-18T23:45:25Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Named entity recognition architecture combining contextual and global features [5.92351086183376]
  Named entity recognition (NER) is an information extraction technique that aims to locate and classify named entities. We propose the combination of contextual features from XLNet and global features from Graph Convolution Network (GCN) to enhance NER performance.
  arXiv  Detail & Related papers  (2021-12-15T10:54:36Z)
• Exploiting Global Contextual Information for Document-level Named Entity Recognition [46.99922251839363]
  We propose a model called Global Context enhanced Document-level NER (GCDoc) At word-level, a document graph is constructed to model a wider range of dependencies between words. At sentence-level, for appropriately modeling wider context beyond single sentence, we employ a cross-sentence module. Our model reaches F1 score of 92.22 (93.40 with BERT) on CoNLL 2003 dataset and 88.32 (90.49 with BERT) on Ontonotes 5.0 dataset.
  arXiv  Detail & Related papers  (2021-06-02T01:52:07Z)
• Global Attention for Name Tagging [56.62059996864408]
  We present a new framework to improve name tagging by utilizing local, document-level, and corpus-level contextual information. We propose a model that learns to incorporate document-level and corpus-level contextual information alongside local contextual information via global attentions. Experiments on benchmark datasets show the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-19T07:27:15Z)
• Global-to-Local Neural Networks for Document-Level Relation Extraction [11.900280120655898]
  Relation extraction (RE) aims to identify the semantic relations between named entities in text. Recent years have witnessed it raised to the document level, which requires complex reasoning with entities and mentions throughout an entire document. We propose a novel model to document-level RE, by encoding the document information in terms of entity global and local representations as well as context relation representations.
  arXiv  Detail & Related papers  (2020-09-22T07:30:19Z)
• Local-Global Video-Text Interactions for Temporal Grounding [77.5114709695216]
  This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query. We tackle this problem using a novel regression-based model that learns to extract a collection of mid-level features for semantic phrases in a text query. The proposed method effectively predicts the target time interval by exploiting contextual information from local to global.
  arXiv  Detail & Related papers  (2020-04-16T08:10:41Z)
• Towards Making the Most of Context in Neural Machine Translation [112.9845226123306]
  We argue that previous research did not make a clear use of the global context. We propose a new document-level NMT framework that deliberately models the local context of each sentence.
  arXiv  Detail & Related papers  (2020-02-19T03:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.