Exploiting Fine-Grained Prototype Distribution for Boosting Unsupervised Class Incremental Learning

• URL: http://arxiv.org/abs/2408.10046v1
• Date: Mon, 19 Aug 2024 14:38:27 GMT
• Title: Exploiting Fine-Grained Prototype Distribution for Boosting Unsupervised Class Incremental Learning
• Authors: Jiaming Liu, Hongyuan Liu, Zhili Qin, Wei Han, Yulu Fan, Qinli Yang, Junming Shao,
• Abstract summary: This paper explores a more challenging problem of unsupervised class incremental learning (UCIL) The essence of addressing this problem lies in effectively capturing comprehensive feature representations and discovering unknown novel classes. We propose a strategy to minimize overlap between novel and existing classes, thereby preserving historical knowledge and mitigating the phenomenon of catastrophic forgetting.
• Score: 13.17775851211893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The dynamic nature of open-world scenarios has attracted more attention to class incremental learning (CIL). However, existing CIL methods typically presume the availability of complete ground-truth labels throughout the training process, an assumption rarely met in practical applications. Consequently, this paper explores a more challenging problem of unsupervised class incremental learning (UCIL). The essence of addressing this problem lies in effectively capturing comprehensive feature representations and discovering unknown novel classes. To achieve this, we first model the knowledge of class distribution by exploiting fine-grained prototypes. Subsequently, a granularity alignment technique is introduced to enhance the unsupervised class discovery. Additionally, we proposed a strategy to minimize overlap between novel and existing classes, thereby preserving historical knowledge and mitigating the phenomenon of catastrophic forgetting. Extensive experiments on the five datasets demonstrate that our approach significantly outperforms current state-of-the-art methods, indicating the effectiveness of the proposed method.

Related papers

• Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric [99.19559537966538]
  DML aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-03T13:44:20Z)
• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• Few-Shot Class-Incremental Learning with Prior Knowledge [94.95569068211195]
  We propose Learning with Prior Knowledge (LwPK) to enhance the generalization ability of the pre-trained model. Experimental results indicate that LwPK effectively enhances the model resilience against catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-02-02T08:05:35Z)
• NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage [3.4806267677524896]
  We propose a novel active learning strategy, neural tangent kernel clustering-pseudo-labels (NTKCPL) It estimates empirical risk based on pseudo-labels and the model prediction with NTK approximation. We validate our method on five datasets, empirically demonstrating that it outperforms the baseline methods in most cases.
  arXiv  Detail & Related papers  (2023-06-07T01:43:47Z)
• Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery [76.63807209414789]
  We challenge the status quo in class-iNCD and propose a learning paradigm where class discovery occurs continuously and truly unsupervisedly. We propose simple baselines, composed of a frozen PTM backbone and a learnable linear classifier, that are not only simple to implement but also resilient under longer learning scenarios.
  arXiv  Detail & Related papers  (2023-03-28T13:47:16Z)
• Mitigating Forgetting in Online Continual Learning via Contrasting Semantically Distinct Augmentations [22.289830907729705]
  Online continual learning (OCL) aims to enable model learning from a non-stationary data stream to continuously acquire new knowledge as well as retain the learnt one. Main challenge comes from the "catastrophic forgetting" issue -- the inability to well remember the learnt knowledge while learning the new ones.
  arXiv  Detail & Related papers  (2022-11-10T05:29:43Z)
• Prototypical quadruplet for few-shot class incremental learning [24.814045065163135]
  We propose a novel method that improves classification robustness by identifying a better embedding space using an improved contrasting loss. Our approach retains previously acquired knowledge in the embedding space, even when trained with new classes. We demonstrate the effectiveness of our method by showing that the embedding space remains intact after training the model with new classes and outperforms existing state-of-the-art algorithms in terms of accuracy across different sessions.
  arXiv  Detail & Related papers  (2022-11-05T17:19:14Z)
• MCDAL: Maximum Classifier Discrepancy for Active Learning [74.73133545019877]
  Recent state-of-the-art active learning methods have mostly leveraged Generative Adversarial Networks (GAN) for sample acquisition. We propose in this paper a novel active learning framework that we call Maximum Discrepancy for Active Learning (MCDAL) In particular, we utilize two auxiliary classification layers that learn tighter decision boundaries by maximizing the discrepancies among them.
  arXiv  Detail & Related papers  (2021-07-23T06:57:08Z)
• A Survey on Deep Semi-supervised Learning [51.26862262550445]
  We first present a taxonomy for deep semi-supervised learning that categorizes existing methods. We then offer a detailed comparison of these methods in terms of the type of losses, contributions, and architecture differences.
  arXiv  Detail & Related papers  (2021-02-28T16:22:58Z)
• A Comprehensive Approach to Unsupervised Embedding Learning based on AND Algorithm [18.670975246545208]
  Unsupervised embedding learning aims to extract good representation from data without the need for any manual labels. This paper proposes a new unsupervised embedding approach, called Super-AND, which extends the current state-of-the-art model. Super-AND outperforms all existing approaches and achieves an accuracy of 89.2% on the image classification task for CIFAR-10.
  arXiv  Detail & Related papers  (2020-02-26T13:22:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.