KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers

• URL: http://arxiv.org/abs/2111.15436v1
• Date: Tue, 30 Nov 2021 14:29:33 GMT
• Title: KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers
• Authors: Avi Chawla, Nidhi Mulay, Vikas Bishnoi, Gaurav Dhama
• Abstract summary: We propose a Knowledge Aware Representation Learning (KARL) Network for Named Entity Recognition (NER) KARL is based on a Transformer that utilizes large knowledge bases represented as fact triplets, converts them to a context, and extracts essential information residing inside to generate contextualized triplet representation for feature augmentation. Experimental results show that the augmentation done using KARL can considerably boost the performance of our NER system and achieve significantly better results than existing approaches in the literature on three publicly available NER datasets, namely CoNLL 2003, CoNLL++, and OntoNotes v5.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inception of modeling contextual information using models such as BERT, ELMo, and Flair has significantly improved representation learning for words. It has also given SOTA results in almost every NLP task - Machine Translation, Text Summarization and Named Entity Recognition, to name a few. In this work, in addition to using these dominant context-aware representations, we propose a Knowledge Aware Representation Learning (KARL) Network for Named Entity Recognition (NER). We discuss the challenges of using existing methods in incorporating world knowledge for NER and show how our proposed methods could be leveraged to overcome those challenges. KARL is based on a Transformer Encoder that utilizes large knowledge bases represented as fact triplets, converts them to a graph context, and extracts essential entity information residing inside to generate contextualized triplet representation for feature augmentation. Experimental results show that the augmentation done using KARL can considerably boost the performance of our NER system and achieve significantly better results than existing approaches in the literature on three publicly available NER datasets, namely CoNLL 2003, CoNLL++, and OntoNotes v5. We also observe better generalization and application to a real-world setting from KARL on unseen entities.

Related papers

• In-Context Learning for Few-Shot Nested Named Entity Recognition [53.55310639969833]
  We introduce an effective and innovative ICL framework for the setting of few-shot nested NER. We improve the ICL prompt by devising a novel example demonstration selection mechanism, EnDe retriever. In EnDe retriever, we employ contrastive learning to perform three types of representation learning, in terms of semantic similarity, boundary similarity, and label similarity.
  arXiv  Detail & Related papers  (2024-02-02T06:57:53Z)
• NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval [49.827932299460514]
  We argue that capabilities provided by large language models are not the end of NER research, but rather an exciting beginning. We present three variants of the NER task, together with a dataset to support them. We provide a large, silver-annotated corpus of 4 million paragraphs covering 500 entity types.
  arXiv  Detail & Related papers  (2023-10-22T12:23:00Z)
• IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition using Knowledge Bases [53.054598423181844]
  We present a novel NER cascade approach comprising three steps. We empirically demonstrate the significance of external knowledge bases in accurately classifying fine-grained and emerging entities. Our system exhibits robust performance in the MultiCoNER2 shared task, even in the low-resource language setting.
  arXiv  Detail & Related papers  (2023-04-20T20:30:34Z)
• MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective [57.19660234992812]
  NER model has achieved promising performance on standard NER benchmarks. Recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. We propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective.
  arXiv  Detail & Related papers  (2022-04-09T05:18:20Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)
• AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction [54.20039200180071]
  We present a novel framework to improve NER performance by automatically generating and leveraging entity triggers'' Our framework leverages post-hoc explanation to generate rationales and strengthens a model's prior knowledge using an embedding technique. AutoTriggER shows strong label-efficiency, is capable of generalizing to unseen entities, and outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on average.
  arXiv  Detail & Related papers  (2021-09-10T08:11:56Z)
• Empirical Study of Named Entity Recognition Performance Using Distribution-aware Word Embedding [15.955385058787348]
  We develop a distribution-aware word embedding and implement three different methods to make use of the distribution information in a NER framework. The performance of NER will be improved if the word specificity is incorporated into existing NER methods.
  arXiv  Detail & Related papers  (2021-09-03T17:28:04Z)
• Improving Named Entity Recognition with Attentive Ensemble of Syntactic Information [36.03316058182617]
  Named entity recognition (NER) is highly sensitive to sentential syntactic and semantic properties. In this paper, we improve NER by leveraging different types of syntactic information through attentive ensemble. Experimental results on six English and Chinese benchmark datasets suggest the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2020-10-29T10:25:17Z)
• ASTRAL: Adversarial Trained LSTM-CNN for Named Entity Recognition [16.43239147870092]
  We propose an Adversarial Trained LSTM-CNN (ASTRAL) system to improve the current NER method from both the model structure and the training process. Our system is evaluated on three benchmarks, CoNLL-03, OntoNotes 5.0, and WNUT-17, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2020-09-02T13:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.