Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning

• URL: http://arxiv.org/abs/2502.01866v1
• Date: Mon, 03 Feb 2025 22:31:36 GMT
• Title: Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning
• Authors: Edoardo Urettini, Antonio Carta,
• Abstract summary: We formalize replay-based online joint optimization with explicit KL-divergence constraints on replay data. We show how to adapt the estimation of the FIM to a continual setting second-order optimization for non-iid data. OCAR outperforms state-of-the-art methods in continual metrics achieving higher average accuracy throughout the training process in three different benchmarks.
• Score: 2.0165668334347187
• License:
• Abstract: Online Continual Learning (OCL) models continuously adapt to nonstationary data streams, usually without task information. These settings are complex and many traditional CL methods fail, while online methods (mainly replay-based) suffer from instabilities after the task shift. To address this issue, we formalize replay-based OCL as a second-order online joint optimization with explicit KL-divergence constraints on replay data. We propose Online Curvature-Aware Replay (OCAR) to solve the problem: a method that leverages second-order information of the loss using a K-FAC approximation of the Fisher Information Matrix (FIM) to precondition the gradient. The FIM acts as a stabilizer to prevent forgetting while also accelerating the optimization in non-interfering directions. We show how to adapt the estimation of the FIM to a continual setting stabilizing second-order optimization for non-iid data, uncovering the role of the Tikhonov regularization in the stability-plasticity tradeoff. Empirical results show that OCAR outperforms state-of-the-art methods in continual metrics achieving higher average accuracy throughout the training process in three different benchmarks.

Related papers

• Large Continual Instruction Assistant [59.585544987096974]
  Continual Instruction Tuning (CIT) is adopted to instruct Large Models to follow human intent data by data. Existing update gradient would heavily destroy the performance on previous datasets during CIT process. We propose a general continual instruction tuning framework to address the challenge.
  arXiv  Detail & Related papers  (2024-10-08T11:24:59Z)
• Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning [13.836798036474143]
  Key challenge in Federated Class Continual Learning is catastrophic forgetting. We propose a novel method of data replay based on diffusion models. Our method significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2024-09-02T10:07:24Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right? [135.71855998537347]
  We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy. We show that this metric is unreliable, as even vacuous blind classifiers can achieve unrealistically high online accuracy. Existing OCL algorithms can also achieve high online accuracy, but perform poorly in retaining useful information.
  arXiv  Detail & Related papers  (2023-05-16T08:29:33Z)
• Online Hyperparameter Optimization for Class-Incremental Learning [99.70569355681174]
  Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. We propose an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori.
  arXiv  Detail & Related papers  (2023-01-11T17:58:51Z)
• Improving information retention in large scale online continual learning [99.73847522194549]
  Online continual learning aims to adapt efficiently to new data while retaining existing knowledge. Recent work suggests that information retention remains a problem in large scale OCL even when the replay buffer is unlimited. We propose using a moving average family of methods to improve optimization for non-stationary objectives.
  arXiv  Detail & Related papers  (2022-10-12T16:59:43Z)
• Offline Reinforcement Learning with Fisher Divergence Critic Regularization [41.085156836450466]
  We propose an alternative approach to encouraging the learned policy to stay close to the data, namely parameterizing the critic as the log-behavior-policy. Behavior regularization then corresponds to an appropriate regularizer on the offset term. Our algorithm Fisher-BRC achieves both improved performance and faster convergence over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-14T22:11:40Z)
• Generalized Variational Continual Learning [33.194866396158005]
  Two main approaches to continuous learning are Online Elastic Weight Consolidation and Variational Continual Learning. We show that applying this modification to mitigate Online EWC as a limiting case, allowing baselines between the two approaches. In order to the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, mitigate neural networks with task-specific FiLM layers.
  arXiv  Detail & Related papers  (2020-11-24T19:07:39Z)
• Tracking Performance of Online Stochastic Learners [57.14673504239551]
  Online algorithms are popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches. When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy. We establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models.
  arXiv  Detail & Related papers  (2020-04-04T14:16:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.