Span-based Named Entity Recognition by Generating and Compressing Information

• URL: http://arxiv.org/abs/2302.05392v1
• Date: Fri, 10 Feb 2023 17:40:51 GMT
• Title: Span-based Named Entity Recognition by Generating and Compressing Information
• Authors: Nhung T.H. Nguyen, Makoto Miwa and Sophia Ananiadou
• Abstract summary: We propose to combine the two types of IB models into one system to enhance Named Entity Recognition. Experiments on five different corpora indicate that jointly training both generative and information compression models can enhance the performance of the baseline span-based NER system.
• Score: 23.444967843156086
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The information bottleneck (IB) principle has been proven effective in various NLP applications. The existing work, however, only used either generative or information compression models to improve the performance of the target task. In this paper, we propose to combine the two types of IB models into one system to enhance Named Entity Recognition (NER). For one type of IB model, we incorporate two unsupervised generative components, span reconstruction and synonym generation, into a span-based NER system. The span reconstruction ensures that the contextualised span representation keeps the span information, while the synonym generation makes synonyms have similar representations even in different contexts. For the other type of IB model, we add a supervised IB layer that performs information compression into the system to preserve useful features for NER in the resulting span representations. Experiments on five different corpora indicate that jointly training both generative and information compression models can enhance the performance of the baseline span-based NER system. Our source code is publicly available at https://github.com/nguyennth/joint-ib-models.

Related papers

• Hybrid Multi-stage Decoding for Few-shot NER with Entity-aware Contrastive Learning [32.62763647036567]
  Few-shot named entity recognition can identify new types of named entities based on a few labeled examples. We propose the Hybrid Multi-stage Decoding for Few-shot NER with Entity-aware Contrastive Learning (MsFNER) MsFNER splits the general NER into two stages: entity-span detection and entity classification.
  arXiv  Detail & Related papers  (2024-04-10T12:31:09Z)
• Neural Coreference Resolution based on Reinforcement Learning [53.73316523766183]
  Coreference resolution systems need to solve two subtasks. One task is to detect all of the potential mentions, the other is to learn the linking of an antecedent for each possible mention. We propose a reinforcement learning actor-critic-based neural coreference resolution system.
  arXiv  Detail & Related papers  (2022-12-18T07:36:35Z)
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression [53.90578309960526]
  Large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. We propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
  arXiv  Detail & Related papers  (2021-10-16T11:23:02Z)
• EchoEA: Echo Information between Entities and Relations for Entity Alignment [1.1470070927586016]
  We propose a novel framework, Echo Entity Alignment (EchoEA), which leverages self-attention mechanism to spread entity information to relations and echo back to entities. The experimental results on three real-world cross-lingual datasets are stable at around 96% at hits@1 on average.
  arXiv  Detail & Related papers  (2021-07-07T07:34:21Z)
• SpanNer: Named Entity Re-/Recognition as Span Prediction [62.66148736099347]
  span prediction model is used for named entity recognition. We experimentally implement 154 systems on 11 datasets, covering three languages. Our model has been deployed into the ExplainaBoard platform.
  arXiv  Detail & Related papers  (2021-06-01T17:11:42Z)
• Better Feature Integration for Named Entity Recognition [30.676768644145]
  We propose a simple and robust solution to incorporate both types of features with our Synergized-LSTM (Syn-LSTM) The results demonstrate that the proposed model achieves better performance than previous approaches while requiring fewer parameters.
  arXiv  Detail & Related papers  (2021-04-12T09:55:06Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Learning from Context or Names? An Empirical Study on Neural Relation Extraction [112.06614505580501]
  We study the effect of two main information sources in text: textual context and entity mentions (names) We propose an entity-masked contrastive pre-training framework for relation extraction (RE) Our framework can improve the effectiveness and robustness of neural models in different RE scenarios.
  arXiv  Detail & Related papers  (2020-10-05T11:21:59Z)
• Bipartite Flat-Graph Network for Nested Named Entity Recognition [94.91507634620133]
  Bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER) We propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER)
  arXiv  Detail & Related papers  (2020-05-01T15:14:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.