Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting

• URL: http://arxiv.org/abs/2304.00933v4
• Date: Mon, 24 Jun 2024 12:03:00 GMT
• Title: Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting
• Authors: Timm Hess, Eli Verwimp, Gido M. van de Ven, Tinne Tuytelaars,
• Abstract summary: We study the coexistence of two phenomena that affect the quality of continually learned representations: knowledge accumulation and feature forgetting. We show that even though forgetting in the representation can be small in absolute terms, forgetting in the representation tends to be just as catastrophic as forgetting at the output level.
• Score: 34.402943976801424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning research has shown that neural networks suffer from catastrophic forgetting "at the output level", but it is debated whether this is also the case at the level of learned representations. Multiple recent studies ascribe representations a certain level of innate robustness against forgetting -- that they only forget minimally in comparison with forgetting at the output level. We revisit and expand upon the experiments that revealed this difference in forgetting and illustrate the coexistence of two phenomena that affect the quality of continually learned representations: knowledge accumulation and feature forgetting. Taking both aspects into account, we show that, even though forgetting in the representation (i.e. feature forgetting) can be small in absolute terms, when measuring relative to how much was learned during a task, forgetting in the representation tends to be just as catastrophic as forgetting at the output level. Next we show that this feature forgetting is problematic as it substantially slows down the incremental learning of good general representations (i.e. knowledge accumulation). Finally, we study how feature forgetting and knowledge accumulation are affected by different types of continual learning methods.

Related papers

• Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex [32.79706360108185]
  We find evidence for such learning in mice following continued training on a task, long after behavior saturates at near-ceiling performance ("overtraining") We demonstrate that class representations in cortex continue to separate during overtraining, so that examples that were incorrectly classified at the beginning of overtraining can abruptly be correctly classified later on, despite no changes in behavior during that time. We conclude by showing how this model of late-time feature learning implies an explanation for the empirical puzzle overtraining reversal in animal learning.
  arXiv  Detail & Related papers  (2024-11-05T22:42:49Z)
• CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning [52.63674911541416]
  Few-shot class-incremental learning (FSCIL) faces several challenges, such as overfitting and forgetting. Our primary focus is representation learning on base classes to tackle the unique challenge of FSCIL. We find that trying to secure the spread of features within a more confined feature space enables the learned representation to strike a better balance between transferability and discriminability.
  arXiv  Detail & Related papers  (2024-10-08T02:23:16Z)
• The Curious Case of Benign Memorization [19.74244993871716]
  We show that under training protocols that include data augmentation, neural networks learn to memorize entirely random labels in a benign way. We demonstrate that deep models have the surprising ability to separate noise from signal by distributing the task of memorization and feature learning to different layers.
  arXiv  Detail & Related papers  (2022-10-25T13:41:31Z)
• Feature Forgetting in Continual Representation Learning [48.89340526235304]
  representations do not suffer from "catastrophic forgetting" even in plain continual learning, but little further fact is known about its characteristics. We devise a protocol for evaluating representation in continual learning, and then use it to present an overview of the basic trends of continual representation learning. To study the feature forgetting problem, we create a synthetic dataset to identify and visualize the prevalence of feature forgetting in neural networks.
  arXiv  Detail & Related papers  (2022-05-26T13:38:56Z)
• Continual Feature Selection: Spurious Features in Continual Learning [0.0]
  This paper studies spurious features' influence on continual learning algorithms. We show that learning algorithms solve tasks by overfitting features that are not generalizable.
  arXiv  Detail & Related papers  (2022-03-02T10:43:54Z)
• Understanding Contrastive Learning Requires Incorporating Inductive Biases [64.56006519908213]
  Recent attempts to theoretically explain the success of contrastive learning on downstream tasks prove guarantees depending on properties of em augmentations and the value of em contrastive loss of representations. We demonstrate that such analyses ignore em inductive biases of the function class and training algorithm, even em provably leading to vacuous guarantees in some settings.
  arXiv  Detail & Related papers  (2022-02-28T18:59:20Z)
• Essentials for Class Incremental Learning [43.306374557919646]
  Class-incremental learning results on CIFAR-100 and ImageNet improve over the state-of-the-art by a large margin, while keeping the approach simple.
  arXiv  Detail & Related papers  (2021-02-18T18:01:06Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)
• Weakly-Supervised Disentanglement Without Compromises [53.55580957483103]
  Intelligent agents should be able to learn useful representations by observing changes in their environment. We model such observations as pairs of non-i.i.d. images sharing at least one of the underlying factors of variation. We show that only knowing how many factors have changed, but not which ones, is sufficient to learn disentangled representations.
  arXiv  Detail & Related papers  (2020-02-07T16:39:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.