Hierarchical Transformer Model for Scientific Named Entity Recognition

• URL: http://arxiv.org/abs/2203.14710v1
• Date: Mon, 28 Mar 2022 12:59:06 GMT
• Title: Hierarchical Transformer Model for Scientific Named Entity Recognition
• Authors: Urchade Zaratiana and Pierre Holat and Nadi Tomeh and Thierry Charnois
• Abstract summary: We present a simple and effective approach for Named Entity Recognition. The main idea of our approach is to encode the input subword sequence with a pre-trained transformer such as BERT. We evaluate our approach on three benchmark datasets for scientific NER.
• Score: 0.20646127669654832
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The task of Named Entity Recognition (NER) is an important component of many natural language processing systems, such as relation extraction and knowledge graph construction. In this work, we present a simple and effective approach for Named Entity Recognition. The main idea of our approach is to encode the input subword sequence with a pre-trained transformer such as BERT, and then, instead of directly classifying the word labels, another layer of transformer is added to the subword representation to better encode the word-level interaction. We evaluate our approach on three benchmark datasets for scientific NER, particularly in the computer science and biomedical domains. Experimental results show that our model outperforms the current state-of-the-art on SciERC and TDM datasets without requiring external resources or specific data augmentation. Code is available at \url{https://github.com/urchade/HNER}.

Related papers

• Named Entity Recognition for Address Extraction in Speech-to-Text Transcriptions Using Synthetic Data [0.0]
  This paper introduces an approach for building a Named Entity Recognition (NER) model built upon a Bidirectional Representations from Transformers (BERT) architecture. This NER model extracts address parts from data acquired from speech-to-text transcriptions. The performance of our NER model, trained solely on synthetic data, is evaluated using small real test dataset.
  arXiv  Detail & Related papers  (2024-02-08T10:29:11Z)
• Disambiguation of Company names via Deep Recurrent Networks [101.90357454833845]
  We propose a Siamese LSTM Network approach to extract -- via supervised learning -- an embedding of company name strings. We analyse how an Active Learning approach to prioritise the samples to be labelled leads to a more efficient overall learning pipeline.
  arXiv  Detail & Related papers  (2023-03-07T15:07:57Z)
• Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning [80.36076044023581]
  We present an efficient bi-encoder framework for named entity recognition (NER) We frame NER as a metric learning problem that maximizes the similarity between the vector representations of an entity mention and its type. A major challenge to this bi-encoder formulation for NER lies in separating non-entity spans from entity mentions.
  arXiv  Detail & Related papers  (2022-08-30T23:19:04Z)
• Label Semantics for Few Shot Named Entity Recognition [68.01364012546402]
  We study the problem of few shot learning for named entity recognition. We leverage the semantic information in the names of the labels as a way of giving the model additional signal and enriched priors. Our model learns to match the representations of named entities computed by the first encoder with label representations computed by the second encoder.
  arXiv  Detail & Related papers  (2022-03-16T23:21:05Z)
• Investigation on Data Adaptation Techniques for Neural Named Entity Recognition [51.88382864759973]
  A common practice is to utilize large monolingual unlabeled corpora. Another popular technique is to create synthetic data from the original labeled data. In this work, we investigate the impact of these two methods on the performance of three different named entity recognition tasks.
  arXiv  Detail & Related papers  (2021-10-12T11:06:03Z)
• BERT got a Date: Introducing Transformers to Temporal Tagging [4.651578365545765]
  We present a transformer encoder-decoder model using the RoBERTa language model as our best performing system. Our model surpasses previous works in temporal tagging and type classification, especially on rare classes.
  arXiv  Detail & Related papers  (2021-09-30T08:54:21Z)
• QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction [57.56700153507383]
  This paper proposes a unified query attribute value extraction system in e-commerce search named QUEACO. For the NER phase, QUEACO adopts a novel teacher-student network, where a teacher network that is trained on the strongly-labeled data generates pseudo-labels. For the AVN phase, we also leverage the weakly-labeled query-to-attribute behavior data to normalize surface form attribute values from queries into canonical forms from products.
  arXiv  Detail & Related papers  (2021-08-19T03:24:23Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• An Analysis of Simple Data Augmentation for Named Entity Recognition [21.013836715832564]
  We design and compare data augmentation for named entity recognition. We show that simple augmentation can boost performance for both recurrent and transformer-based models.
  arXiv  Detail & Related papers  (2020-10-22T13:21:03Z)
• Neural Entity Linking on Technical Service Tickets [1.3621712165154805]
  We show that a neural approach outperforms and complements hand-coded entities, with improvements of about 20% top-1 accuracy. We also show that a simple sentence-wise encoding (Bi-Encoder) offers a fast yet efficient search in practice.
  arXiv  Detail & Related papers  (2020-05-15T15:47:02Z)
• TNT-KID: Transformer-based Neural Tagger for Keyword Identification [7.91883337742071]
  We present a novel algorithm for keyword identification called Transformer-based Neural Tagger for Keyword IDentification (TNT-KID) By adapting the transformer architecture for a specific task at hand and leveraging language model pretraining on a domain specific corpus, the model is capable of overcoming deficiencies of both supervised and unsupervised state-of-the-art approaches to keyword extraction.
  arXiv  Detail & Related papers  (2020-03-20T09:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.