BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision

• URL: http://arxiv.org/abs/2006.15509v1
• Date: Sun, 28 Jun 2020 04:55:39 GMT
• Title: BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
• Authors: Chen Liang, Yue Yu, Haoming Jiang, Siawpeng Er, Ruijia Wang, Tuo Zhao, Chao Zhang
• Abstract summary: We propose a new computational framework -- BOND -- to improve the prediction performance of NER models. Specifically, we propose a two-stage training algorithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels. In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance.
• Score: 49.42215511723874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the open-domain named entity recognition (NER) problem under distant supervision. The distant supervision, though does not require large amounts of manual annotations, yields highly incomplete and noisy distant labels via external knowledge bases. To address this challenge, we propose a new computational framework -- BOND, which leverages the power of pre-trained language models (e.g., BERT and RoBERTa) to improve the prediction performance of NER models. Specifically, we propose a two-stage training algorithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels, which can significantly improve the recall and precision; In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance. Thorough experiments on 5 benchmark datasets demonstrate the superiority of BOND over existing distantly supervised NER methods. The code and distantly labeled data have been released in https://github.com/cliang1453/BOND.

Related papers

• SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entities [10.193908215351497]
  We propose SCANNER, a model capable of effectively handling all three NER variants. SCANNER is a two-stage structure; we extract entity candidates in the first stage and use it as a query to get knowledge. To tackle the challenges arising from noisy annotations in NER datasets, we introduce a novel self-distillation method.
  arXiv  Detail & Related papers  (2024-04-02T13:05:41Z)
• Enhancing Few-shot NER with Prompt Ordering based Data Augmentation [59.69108119752584]
  We propose a Prompt Ordering based Data Augmentation (PODA) method to improve the training of unified autoregressive generation frameworks. Experimental results on three public NER datasets and further analyses demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-05-19T16:25:43Z)
• Gaussian Prior Reinforcement Learning for Nested Named Entity Recognition [52.46740830977898]
  We propose a novel seq2seq model named GPRL, which formulates the nested NER task as an entity triplet sequence generation process. Experiments on three nested NER datasets demonstrate that GPRL outperforms previous nested NER models.
  arXiv  Detail & Related papers  (2023-05-12T05:55:34Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)
• Named Entity Recognition in the Style of Object Detection [5.228551526328475]
  We propose a two-stage method for named entity recognition (NER) First, a region proposal network generates region candidates and then a second-stage model discriminates and classifies the entity and makes the final prediction. We tested the model on the nested named entity recognition task ACE2005 and Genia, and got F1 score of 85.6$%$ and 76.8$%$ respectively.
  arXiv  Detail & Related papers  (2021-01-26T22:47:05Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Coarse-to-Fine Pre-training for Named Entity Recognition [26.00489191164784]
  We propose a NER-specific pre-training framework to in-ject coarse-to-fine automatically mined entityknowledge into pre-trained models. Our framework achieves significant improvements against several pre-trained base-lines, establishing the new state-of-the-art per-formance on three benchmarks.
  arXiv  Detail & Related papers  (2020-10-16T07:39:20Z)
• Named Entity Recognition without Labelled Data: A Weak Supervision Approach [23.05371427663683]
  This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. A sequence labelling model can finally be trained on the basis of this unified annotation.
  arXiv  Detail & Related papers  (2020-04-30T12:29:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.