How to tackle an emerging topic? Combining strong and weak labels for Covid news NER

• URL: http://arxiv.org/abs/2209.15108v1
• Date: Thu, 29 Sep 2022 21:33:02 GMT
• Title: How to tackle an emerging topic? Combining strong and weak labels for Covid news NER
• Authors: Aleksander Ficek, Fangyu Liu, Nigel Collier
• Abstract summary: We introduce a novel COVID-19 news NER dataset (COVIDNEWS-NER) We release 3000 entries of hand annotated strongly labelled sentences and 13000 auto-generated weakly labelled sentences. We show the effectiveness of CONTROSTER on COVIDNEWS-NER while providing analysis on combining weak and strong labels for training.
• Score: 90.90053968189156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Being able to train Named Entity Recognition (NER) models for emerging topics is crucial for many real-world applications especially in the medical domain where new topics are continuously evolving out of the scope of existing models and datasets. For a realistic evaluation setup, we introduce a novel COVID-19 news NER dataset (COVIDNEWS-NER) and release 3000 entries of hand annotated strongly labelled sentences and 13000 auto-generated weakly labelled sentences. Besides the dataset, we propose CONTROSTER, a recipe to strategically combine weak and strong labels in improving NER in an emerging topic through transfer learning. We show the effectiveness of CONTROSTER on COVIDNEWS-NER while providing analysis on combining weak and strong labels for training. Our key findings are: (1) Using weak data to formulate an initial backbone before tuning on strong data outperforms methods trained on only strong or weak data. (2) A combination of out-of-domain and in-domain weak label training is crucial and can overcome saturation when being training on weak labels from a single source.

Related papers

• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Label Noise-Resistant Mean Teaching for Weakly Supervised Fake News Detection [93.6222609806278]
  We propose a novel label noise-resistant mean teaching approach (LNMT) for weakly supervised fake news detection. LNMT leverages unlabeled news and feedback comments of users to enlarge the amount of training data. LNMT establishes a mean teacher framework equipped with label propagation and label reliability estimation.
  arXiv  Detail & Related papers  (2022-06-10T16:01:58Z)
• Data Consistency for Weakly Supervised Learning [15.365232702938677]
  Training machine learning models involves using large amounts of human-annotated data. We propose a novel weak supervision algorithm that processes noisy labels, i.e., weak signals. We show that it significantly outperforms state-of-the-art weak supervision methods on both text and image classification tasks.
  arXiv  Detail & Related papers  (2022-02-08T16:48:19Z)
• Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data [37.980010197914105]
  Weak supervision has shown promising results in many natural language processing tasks, such as Named Entity Recognition (NER) We propose a new multi-stage computational framework -- NEEDLE with three essential ingredients: weak label completion, noise-aware loss function, and final fine-tuning over the strongly labeled data. We demonstrate that NEEDLE can effectively suppress the noise of the weak labels and outperforms existing methods.
  arXiv  Detail & Related papers  (2021-06-16T17:18:14Z)
• Self-Training with Weak Supervision [32.68342091430266]
  State-of-the-art deep neural networks require large-scale labeled training data that is often expensive to obtain or not available for many tasks. weak supervision in the form of domain-specific rules has been shown to be useful in such settings. We develop a weak supervision framework (ASTRA) that leverages all the available data for a given task.
  arXiv  Detail & Related papers  (2021-04-12T14:45:04Z)
• Social Adaptive Module for Weakly-supervised Group Activity Recognition [143.68241396839062]
  This paper presents a new task named weakly-supervised group activity recognition (GAR) It differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data. This eases us to collect and annotate a large-scale NBA dataset and thus raise new challenges to GAR.
  arXiv  Detail & Related papers  (2020-07-18T16:40:55Z)
• Temporal Calibrated Regularization for Robust Noisy Label Learning [60.90967240168525]
  Deep neural networks (DNNs) exhibit great success on many tasks with the help of large-scale well annotated datasets. However, labeling large-scale data can be very costly and error-prone so that it is difficult to guarantee the annotation quality. We propose a Temporal Calibrated Regularization (TCR) in which we utilize the original labels and the predictions in the previous epoch together.
  arXiv  Detail & Related papers  (2020-07-01T04:48:49Z)
• Classify and Generate Reciprocally: Simultaneous Positive-Unlabelled Learning and Conditional Generation with Extra Data [77.31213472792088]
  The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems. We address this problem by leveraging Positive-Unlabeled(PU) classification and the conditional generation with extra unlabeled data. We present a novel training framework to jointly target both PU classification and conditional generation when exposed to extra data.
  arXiv  Detail & Related papers  (2020-06-14T08:27:40Z)
• Named Entity Recognition without Labelled Data: A Weak Supervision Approach [23.05371427663683]
  This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. A sequence labelling model can finally be trained on the basis of this unified annotation.
  arXiv  Detail & Related papers  (2020-04-30T12:29:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.