Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning

• URL: http://arxiv.org/abs/2110.04429v1
• Date: Sat, 9 Oct 2021 01:45:03 GMT
• Title: Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning
• Authors: Xinghua Zhang, Bowen Yu, Tingwen Liu, Zhenyu Zhang, Jiawei Sheng, Mengge Xue, Hongbo Xu
• Abstract summary: We propose a robust learning paradigm named Self-Collaborative Denoising Learning (SCDL) SCDL jointly trains two teacher-student networks in a mutually-beneficial manner to iteratively perform noisy label refinery. Extensive experimental results on five real-world datasets demonstrate that SCDL is superior to state-of-the-art DS-NER denoising methods.
• Score: 9.747173655999427
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distantly supervised named entity recognition (DS-NER) efficiently reduces labor costs but meanwhile intrinsically suffers from the label noise due to the strong assumption of distant supervision. Typically, the wrongly labeled instances comprise numbers of incomplete and inaccurate annotation noise, while most prior denoising works are only concerned with one kind of noise and fail to fully explore useful information in the whole training set. To address this issue, we propose a robust learning paradigm named Self-Collaborative Denoising Learning (SCDL), which jointly trains two teacher-student networks in a mutually-beneficial manner to iteratively perform noisy label refinery. Each network is designed to exploit reliable labels via self denoising, and two networks communicate with each other to explore unreliable annotations by collaborative denoising. Extensive experimental results on five real-world datasets demonstrate that SCDL is superior to state-of-the-art DS-NER denoising methods.

Related papers

• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• DIDA: Denoised Imitation Learning based on Domain Adaptation [28.36684781402964]
  We focus on the problem of Learning from Noisy Demonstrations (LND), where the imitator is required to learn from data with noise. We propose Denoised Imitation learning based on Domain Adaptation (DIDA), which designs two discriminators to distinguish the noise level and expertise level of data. Experiment results on MuJoCo demonstrate that DIDA can successfully handle challenging imitation tasks from demonstrations with various types of noise, outperforming most baseline methods.
  arXiv  Detail & Related papers  (2024-04-04T11:29:05Z)
• Improving the Robustness of Distantly-Supervised Named Entity Recognition via Uncertainty-Aware Teacher Learning and Student-Student Collaborative Learning [24.733773208117363]
  We propose Uncertainty-Aware Teacher Learning to reduce the number of incorrect pseudo labels in the self-training stage. We also propose Student-Student Collaborative Learning that allows the transfer of reliable labels between two student networks. We evaluate our proposed method on five DS-NER datasets, demonstrating that our method is superior to the state-of-the-art DS-NER methods.
  arXiv  Detail & Related papers  (2023-11-14T09:09:58Z)
• Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble [56.705249154629264]
  Self-training teacher-student frameworks are proposed to improve the robustness of NER models. In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks. Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
  arXiv  Detail & Related papers  (2022-12-13T12:14:09Z)
• ConNER: Consistency Training for Cross-lingual Named Entity Recognition [96.84391089120847]
  Cross-lingual named entity recognition suffers from data scarcity in the target languages. We propose ConNER as a novel consistency training framework for cross-lingual NER.
  arXiv  Detail & Related papers  (2022-11-17T07:57:54Z)
• Towards Harnessing Feature Embedding for Robust Learning with Noisy Labels [44.133307197696446]
  The memorization effect of deep neural networks (DNNs) plays a pivotal role in recent label noise learning methods. We propose a novel feature embedding-based method for deep learning with label noise, termed LabEl NoiseDilution (LEND)
  arXiv  Detail & Related papers  (2022-06-27T02:45:09Z)
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations [54.400167806154535]
  Existing research on learning with noisy labels mainly focuses on synthetic label noise. This work presents two new benchmark datasets (CIFAR-10N, CIFAR-100N) We show that real-world noisy labels follow an instance-dependent pattern rather than the classically adopted class-dependent ones.
  arXiv  Detail & Related papers  (2021-10-22T22:42:11Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• Noisy Concurrent Training for Efficient Learning under Label Noise [13.041607703862724]
  Deep neural networks (DNNs) fail to learn effectively under label noise and have been shown to memorize random labels which affect their performance. We consider learning in isolation, using one-hot encoded labels as the sole source of supervision, and a lack of regularization to discourage memorization as the major shortcomings of the standard training procedure. We propose Noisy Concurrent Training (NCT) which leverages collaborative learning to use the consensus between two models as an additional source of supervision.
  arXiv  Detail & Related papers  (2020-09-17T14:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.