Incremental Learning with Repetition via Pseudo-Feature Projection

• URL: http://arxiv.org/abs/2502.19922v1
• Date: Thu, 27 Feb 2025 09:43:35 GMT
• Title: Incremental Learning with Repetition via Pseudo-Feature Projection
• Authors: Benedikt Tscheschner, Eduardo Veas, Marc Masana,
• Abstract summary: We investigate how exemplar-free incremental learning strategies are affected by data repetition.<n>Our proposed exemplar-free method achieves competitive results in the classic scenario without repetition, and state-of-the-art performance in the one with repetition.
• Score: 3.4734633097581815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Incremental Learning scenarios do not always represent real-world inference use-cases, which tend to have less strict task boundaries, and exhibit repetition of common classes and concepts in their continual data stream. To better represent these use-cases, new scenarios with partial repetition and mixing of tasks are proposed, where the repetition patterns are innate to the scenario and unknown to the strategy. We investigate how exemplar-free incremental learning strategies are affected by data repetition, and we adapt a series of state-of-the-art approaches to analyse and fairly compare them under both settings. Further, we also propose a novel method (Horde), able to dynamically adjust an ensemble of self-reliant feature extractors, and align them by exploiting class repetition. Our proposed exemplar-free method achieves competitive results in the classic scenario without repetition, and state-of-the-art performance in the one with repetition.

Related papers

• Few-Shot, No Problem: Descriptive Continual Relation Extraction [27.296604792388646]
  Few-shot Continual Relation Extraction is a crucial challenge for enabling AI systems to identify and adapt to evolving relationships in real-world domains. Traditional memory-based approaches often overfit to limited samples, failing to reinforce old knowledge. We propose a novel retrieval-based solution, starting with a large language model to generate descriptions for each relation.
  arXiv  Detail & Related papers  (2025-02-27T23:44:30Z)
• ReconBoost: Boosting Can Achieve Modality Reconcilement [89.4377895465204]
  We study the modality-alternating learning paradigm to achieve reconcilement. We propose a new method called ReconBoost to update a fixed modality each time. We show that the proposed method resembles Friedman's Gradient-Boosting (GB) algorithm, where the updated learner can correct errors made by others.
  arXiv  Detail & Related papers  (2024-05-15T13:22:39Z)
• Continual Learning in the Presence of Repetition [29.03044158045849]
  Continual learning (CL) provides a framework for training models in ever-evolving environments.<n>The concept of repetition in the data stream is not often considered in standard benchmarks for CL.<n>This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning.
  arXiv  Detail & Related papers  (2024-05-07T08:15:48Z)
• GUIDE: Guidance-based Incremental Learning with Diffusion Models [3.046689922445082]
  We introduce GUIDE, a novel continual learning approach that directs diffusion models to rehearse samples at risk of being forgotten. Our experimental results show that GUIDE significantly reduces catastrophic forgetting, outperforming conventional random sampling approaches and surpassing recent state-of-the-art methods in continual learning with generative replay.
  arXiv  Detail & Related papers  (2024-03-06T18:47:32Z)
• Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical [66.57396042747706]
  Complementary-label learning is a weakly supervised learning problem. We propose a consistent approach that does not rely on the uniform distribution assumption. We find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems.
  arXiv  Detail & Related papers  (2023-11-27T02:59:17Z)
• Class-Incremental Learning with Repetition [17.89286445250716]
  We focus on the family of Class-Incremental with Repetition (CIR) scenario, where repetition is embedded in the definition of the stream. We propose two stream generators that produce a wide range of CIR streams starting from a single dataset and a few interpretable parameters. We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream.
  arXiv  Detail & Related papers  (2023-01-26T20:19:10Z)
• Contrastive Learning for Fair Representations [50.95604482330149]
  Trained classification models can unintentionally lead to biased representations and predictions. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
  arXiv  Detail & Related papers  (2021-09-22T10:47:51Z)
• An Investigation of Replay-based Approaches for Continual Learning [79.0660895390689]
  Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF) Several solution classes have been proposed, of which so-called replay-based approaches seem very promising due to their simplicity and robustness. We empirically investigate replay-based approaches of continual learning and assess their potential for applications.
  arXiv  Detail & Related papers  (2021-08-15T15:05:02Z)
• Open-Set Representation Learning through Combinatorial Embedding [62.05670732352456]
  We are interested in identifying novel concepts in a dataset through representation learning based on the examples in both labeled and unlabeled classes. We propose a learning approach, which naturally clusters examples in unseen classes using the compositional knowledge given by multiple supervised meta-classifiers on heterogeneous label spaces. The proposed algorithm discovers novel concepts via a joint optimization of enhancing the discrimitiveness of unseen classes as well as learning the representations of known classes generalizable to novel ones.
  arXiv  Detail & Related papers  (2021-06-29T11:51:57Z)
• Co$^2$L: Contrastive Continual Learning [69.46643497220586]
  Recent breakthroughs in self-supervised learning show that such algorithms learn visual representations that can be transferred better to unseen tasks. We propose a rehearsal-based continual learning algorithm that focuses on continually learning and maintaining transferable representations.
  arXiv  Detail & Related papers  (2021-06-28T06:14:38Z)
• Learning Invariant Representation for Continual Learning [5.979373021392084]
  A key challenge in Continual learning is catastrophically forgetting previously learned tasks when the agent faces a new one. We propose a new pseudo-rehearsal-based method, named learning Invariant Representation for Continual Learning (IRCL) Disentangling the shared invariant representation helps to learn continually a sequence of tasks, while being more robust to forgetting and having better knowledge transfer.
  arXiv  Detail & Related papers  (2021-01-15T15:12:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.