Learning "O" Helps for Learning More: Handling the Concealed Entity Problem for Class-incremental NER

• URL: http://arxiv.org/abs/2210.04676v2
• Date: Mon, 24 Jul 2023 09:00:03 GMT
• Title: Learning "O" Helps for Learning More: Handling the Concealed Entity Problem for Class-incremental NER
• Authors: Ruotian Ma, Xuanting Chen, Lin Zhang, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yunwen Chen
• Abstract summary: "Unlabeled Entity Problem" leads to severe confusion between "O" and entities. We propose an entity-aware contrastive learning method that adaptively detects entity clusters in "O" We introduce a more realistic and challenging benchmark for class-incremental NER.
• Score: 23.625741716498037
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the categories of named entities rapidly increase, the deployed NER models are required to keep updating toward recognizing more entity types, creating a demand for class-incremental learning for NER. Considering the privacy concerns and storage constraints, the standard paradigm for class-incremental NER updates the models with training data only annotated with the new classes, yet the entities from other entity classes are unlabeled, regarded as "Non-entity" (or "O"). In this work, we conduct an empirical study on the "Unlabeled Entity Problem" and find that it leads to severe confusion between "O" and entities, decreasing class discrimination of old classes and declining the model's ability to learn new classes. To solve the Unlabeled Entity Problem, we propose a novel representation learning method to learn discriminative representations for the entity classes and "O". Specifically, we propose an entity-aware contrastive learning method that adaptively detects entity clusters in "O". Furthermore, we propose two effective distance-based relabeling strategies for better learning the old classes. We introduce a more realistic and challenging benchmark for class-incremental NER, and the proposed method achieves up to 10.62\% improvement over the baseline methods.

Related papers

• Towards Non-Exemplar Semi-Supervised Class-Incremental Learning [33.560003528712414]
  Class-incremental learning aims to gradually recognize new classes while maintaining the discriminability of old ones. We propose a non-exemplar semi-supervised CIL framework with contrastive learning and semi-supervised incremental prototype classifier (Semi-IPC) Semi-IPC learns a prototype for each class with unsupervised regularization, enabling the model to incrementally learn from partially labeled new data.
  arXiv  Detail & Related papers  (2024-03-27T06:28:19Z)
• Continual Named Entity Recognition without Catastrophic Forgetting [37.316700599440935]
  We introduce a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones. We develop a confidence-based pseudo-labeling for the non-entity type. We suggest an adaptive re-weighting type-balanced learning strategy to handle the issue of biased type distribution.
  arXiv  Detail & Related papers  (2023-10-23T03:45:30Z)
• Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition [23.25929285468311]
  Learning Other-Class in the same way as new entity types amplifies the catastrophic forgetting and leads to a substantial performance drop. We propose a unified causal framework to retrieve the causality from both new entity types and Other-Class. Experimental results on three benchmark datasets show that our method outperforms the state-of-the-art method by a large margin.
  arXiv  Detail & Related papers  (2022-10-08T09:37:06Z)
• Class-incremental Novel Class Discovery [76.35226130521758]
  We study the new task of class-incremental Novel Class Discovery (class-iNCD) We propose a novel approach for class-iNCD which prevents forgetting of past information about the base classes. Our experiments, conducted on three common benchmarks, demonstrate that our method significantly outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-18T13:49:27Z)
• Multi-Granularity Regularized Re-Balancing for Class Incremental Learning [32.52884416761171]
  Deep learning models suffer from catastrophic forgetting when learning new tasks. Data imbalance between old and new classes is a key issue that leads to performance degradation of the model. We propose an assumption-agnostic method, Multi-Granularity Regularized re-Balancing, to address this problem.
  arXiv  Detail & Related papers  (2022-06-30T11:04:51Z)
• The Overlooked Classifier in Human-Object Interaction Recognition [82.20671129356037]
  We encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. We propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin.
  arXiv  Detail & Related papers  (2022-03-10T23:35:00Z)
• Novel Class Discovery in Semantic Segmentation [104.30729847367104]
  We introduce a new setting of Novel Class Discovery in Semantic (NCDSS) It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image. We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
  arXiv  Detail & Related papers  (2021-12-03T13:31:59Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training [66.80558875393565]
  We study the problem of training named entity recognition (NER) models using only distantly-labeled data. We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step. Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
  arXiv  Detail & Related papers  (2021-09-10T17:19:56Z)
• Learning Adaptive Embedding Considering Incremental Class [55.21855842960139]
  Class-Incremental Learning (CIL) aims to train a reliable model with the streaming data, which emerges unknown classes sequentially. Different from traditional closed set learning, CIL has two main challenges: 1) Novel class detection. After the novel classes are detected, the model needs to be updated without re-training using entire previous data.
  arXiv  Detail & Related papers  (2020-08-31T04:11:24Z)
• Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels [35.586031601299034]
  We present a technique implementing class-incremental learning for semantic segmentation without using the labeled data the model was initially trained on. We show how to overcome these problems with a novel class-incremental learning technique, which nonetheless requires labels only for the new classes. We evaluate our method on the Cityscapes dataset, where we exceed the mIoU performance of all baselines by 3.5% absolute reaching a result.
  arXiv  Detail & Related papers  (2020-05-12T21:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.