Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery

• URL: http://arxiv.org/abs/2504.08550v1
• Date: Fri, 11 Apr 2025 14:01:49 GMT
• Title: Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery
• Authors: Alireza Fathalizadeh, Roozbeh Razavi-Far,
• Abstract summary: Continual generalized category discovery aims to continuously discover and learn novel categories in incoming data batches.<n>We propose a novel method that integrates Extreme Value Theory with proxy anchors to define boundaries around proxies.<n>We also introduce a novel EVT-based loss function to enhance the learned representation.
• Score: 2.693342141713236
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual generalized category discovery has been introduced and studied in the literature as a method that aims to continuously discover and learn novel categories in incoming data batches while avoiding catastrophic forgetting of previously learned categories. A key component in addressing this challenge is the model's ability to separate novel samples, where Extreme Value Theory (EVT) has been effectively employed. In this work, we propose a novel method that integrates EVT with proxy anchors to define boundaries around proxies using a probability of inclusion function, enabling the rejection of unknown samples. Additionally, we introduce a novel EVT-based loss function to enhance the learned representation, achieving superior performance compared to other deep-metric learning methods in similar settings. Using the derived probability functions, novel samples are effectively separated from previously known categories. However, category discovery within these novel samples can sometimes overestimate the number of new categories. To mitigate this issue, we propose a novel EVT-based approach to reduce the model size and discard redundant proxies. We also incorporate experience replay and knowledge distillation mechanisms during the continual learning stage to prevent catastrophic forgetting. Experimental results demonstrate that our proposed approach outperforms state-of-the-art methods in continual generalized category discovery scenarios.

Related papers

• Generalized Semantic Contrastive Learning via Embedding Side Information for Few-Shot Object Detection [52.490375806093745]
  The objective of few-shot object detection (FSOD) is to detect novel objects with few training samples.<n>We introduce the side information to alleviate the negative influences derived from the feature space and sample viewpoints.<n>Our model outperforms the previous state-of-the-art methods, significantly improving the ability of FSOD in most shots/splits.
  arXiv  Detail & Related papers  (2025-04-09T17:24:05Z)
• CONCLAD: COntinuous Novel CLAss Detector [5.857367484128867]
  We present a comprehensive solution to the problem of continual novel class detection in post-deployment data.<n>We employ an iterative uncertainty estimation algorithm to differentiate between known and novel class(es) samples.<n>We will release our code upon acceptance.
  arXiv  Detail & Related papers  (2024-12-13T01:41:28Z)
• Exploiting Fine-Grained Prototype Distribution for Boosting Unsupervised Class Incremental Learning [13.17775851211893]
  This paper explores a more challenging problem of unsupervised class incremental learning (UCIL) The essence of addressing this problem lies in effectively capturing comprehensive feature representations and discovering unknown novel classes. We propose a strategy to minimize overlap between novel and existing classes, thereby preserving historical knowledge and mitigating the phenomenon of catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-08-19T14:38:27Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Proxy Anchor-based Unsupervised Learning for Continuous Generalized Category Discovery [22.519873617950662]
  We propose a novel unsupervised class incremental learning approach for discovering novel categories on unlabeled sets. The proposed method fine-tunes the feature extractor and proxy anchors on labeled sets, then splits samples into old and novel categories and clusters on the unlabeled dataset. Experimental results demonstrate that our proposed approach outperforms the state-of-the-art methods on fine-grained datasets under real-world scenarios.
  arXiv  Detail & Related papers  (2023-07-20T15:13:29Z)
• Memorizing Complementation Network for Few-Shot Class-Incremental Learning [109.4206979528375]
  We propose a Memorizing Complementation Network (MCNet) to ensemble multiple models that complements the different memorized knowledge with each other in novel tasks. We develop a Prototype Smoothing Hard-mining Triplet (PSHT) loss to push the novel samples away from not only each other in current task but also the old distribution.
  arXiv  Detail & Related papers  (2022-08-11T02:32:41Z)
• Novel Class Discovery without Forgetting [72.52222295216062]
  We identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting. We propose a machine learning model to incrementally discover novel categories of instances from unlabeled data. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery.
  arXiv  Detail & Related papers  (2022-07-21T17:54:36Z)
• Class-incremental Novel Class Discovery [76.35226130521758]
  We study the new task of class-incremental Novel Class Discovery (class-iNCD) We propose a novel approach for class-iNCD which prevents forgetting of past information about the base classes. Our experiments, conducted on three common benchmarks, demonstrate that our method significantly outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-18T13:49:27Z)
• Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning [29.473503894240096]
  We focus on the use of the categorical cross-entropy loss to model data that is not strictly categorical, but rather takes values on the simplex. This practice is standard in neural network architectures with label smoothing and actor-mimic reinforcement learning, amongst others. We propose probabilistically-inspired alternatives to these models, providing an approach that is more principled and theoretically appealing.
  arXiv  Detail & Related papers  (2020-11-10T16:44:35Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)
• Simultaneous Perturbation Stochastic Approximation for Few-Shot Learning [0.5801044612920815]
  We propose a prototypical-like few-shot learning approach based on the prototypical networks method. The results of experiments on the benchmark dataset demonstrate that the proposed method is superior to the original networks.
  arXiv  Detail & Related papers  (2020-06-09T09:47:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.