Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning

• URL: http://arxiv.org/abs/2502.01866v1
• Date: Mon, 03 Feb 2025 22:31:36 GMT
• Title: Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning
• Authors: Edoardo Urettini, Antonio Carta,
• Abstract summary: We formalize replay-based online joint optimization with explicit KL-divergence constraints on replay data.<n>We show how to adapt the estimation of the FIM to a continual setting second-order optimization for non-iid data.<n>OCAR outperforms state-of-the-art methods in continual metrics achieving higher average accuracy throughout the training process in three different benchmarks.
• Score: 2.0165668334347187
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Online Continual Learning (OCL) models continuously adapt to nonstationary data streams, usually without task information. These settings are complex and many traditional CL methods fail, while online methods (mainly replay-based) suffer from instabilities after the task shift. To address this issue, we formalize replay-based OCL as a second-order online joint optimization with explicit KL-divergence constraints on replay data. We propose Online Curvature-Aware Replay (OCAR) to solve the problem: a method that leverages second-order information of the loss using a K-FAC approximation of the Fisher Information Matrix (FIM) to precondition the gradient. The FIM acts as a stabilizer to prevent forgetting while also accelerating the optimization in non-interfering directions. We show how to adapt the estimation of the FIM to a continual setting stabilizing second-order optimization for non-iid data, uncovering the role of the Tikhonov regularization in the stability-plasticity tradeoff. Empirical results show that OCAR outperforms state-of-the-art methods in continual metrics achieving higher average accuracy throughout the training process in three different benchmarks.

Related papers

• Unsupervised Continual Learning for Amortized Bayesian Inference [7.052272974286418]
  Amortized Bayesian Inference (ABI) enables efficient posterior estimation using generative neural networks trained on simulated data.<n>Current approaches are limited to static, single-task settings and fail to handle sequentially arriving data or distribution shifts.<n>We propose a continual learning framework for ABI that decouples simulation-based pre-training from unsupervised sequential SC fine-tuning on real-world data.
  arXiv  Detail & Related papers  (2026-02-26T11:22:46Z)
• Online Continual Learning for Time Series: a Natural Score-driven Approach [2.8989185098518626]
  Online continual learning (OCL) methods adapt to changing environments without forgetting past knowledge.<n>Online time series forecasting (OTSF) is a real-world problem where data evolve in time and success depends on both rapid adaptation and long-term memory.<n>This paper aims to strengthen the theoretical and practical connections between time series methods and OCL.
  arXiv  Detail & Related papers  (2026-01-19T10:31:01Z)
• FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning [63.20028888397869]
  FOREVER (FORgEtting curVe-inspired mEmory) is a novel framework that aligns replay schedules with a model-centric notion of time.<n>Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay.
  arXiv  Detail & Related papers  (2026-01-07T13:55:14Z)
• Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning [29.513882808306406]
  We introduce the Adaptive Replay Buffer (ARB), a novel approach that prioritizes data sampling based on a lightweight metric we call 'on-policyness'<n>ARB is designed to be learning-free and simple to implement, seamlessly integrating into existing Offline-to-Online Reinforcement Learning algorithms.<n>Our experiments on D4RL benchmarks demonstrate that ARB consistently mitigates early performance degradation and significantly improves the final performance of various O2O RL algorithms.
  arXiv  Detail & Related papers  (2025-12-11T10:30:04Z)
• Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning [51.07663354001582]
  Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task.<n>We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches.<n>We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods.
  arXiv  Detail & Related papers  (2025-12-01T15:56:00Z)
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices [61.361819972410046]
  We show why and under what conditions the true sequence-level reward can be optimized via a surrogate token-level objective in policy gradient methods such as REINFORCE.<n>This insight provides a principled explanation for the crucial role of several widely adopted techniques in stabilizing RL training.
  arXiv  Detail & Related papers  (2025-12-01T07:45:39Z)
• Unbiased Online Curvature Approximation for Regularized Graph Continual Learning [9.70311578832594]
  Graph continual learning (GCL) aims to learn from a continuous sequence of graph-based tasks.<n>Regularization methods are vital for preventing catastrophic forgetting in GCL.<n>We propose a new unbiased online curvature approximation of the full Fisher information matrix (FIM) based on the model's current learning state.
  arXiv  Detail & Related papers  (2025-09-16T06:35:13Z)
• From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients [24.963242232471426]
  Continual Learning (CL) aims to learn from a non-stationary data stream where the underlying distribution changes over time.<n>Online CL (onCL) remains dominated by memory-based approaches.<n>We introduce Fine-Grained Hypergradients, an online mechanism for rebalancing gradient updates during training.
  arXiv  Detail & Related papers  (2025-02-26T02:43:54Z)
• TS-ACL: A Time Series Analytic Continual Learning Framework for Privacy-Preserving and Class-Incremental Pattern Recognition [14.108911377558242]
  TS-ACL is a novel framework for privacy-preserving and class-incremental pattern recognition. It transforms each update of the model into a gradient-free analytical learning process with a closed-form solution. It simultaneously achieves non-forgetting, privacy preservation, and lightweight consumption, making it widely suitable for various applications.
  arXiv  Detail & Related papers  (2024-10-21T12:34:02Z)
• Large Continual Instruction Assistant [59.585544987096974]
  Continual Instruction Tuning (CIT) is adopted to instruct Large Models to follow human intent data by data. Existing update gradient would heavily destroy the performance on previous datasets during CIT process. We propose a general continual instruction tuning framework to address the challenge.
  arXiv  Detail & Related papers  (2024-10-08T11:24:59Z)
• Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning [13.836798036474143]
  Key challenge in Federated Class Continual Learning is catastrophic forgetting. We propose a novel method of data replay based on diffusion models. Our method significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2024-09-02T10:07:24Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right? [135.71855998537347]
  We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy. We show that this metric is unreliable, as even vacuous blind classifiers can achieve unrealistically high online accuracy. Existing OCL algorithms can also achieve high online accuracy, but perform poorly in retaining useful information.
  arXiv  Detail & Related papers  (2023-05-16T08:29:33Z)
• Re-Evaluating LiDAR Scene Flow for Autonomous Driving [80.37947791534985]
  Popular benchmarks for self-supervised LiDAR scene flow have unrealistic rates of dynamic motion, unrealistic correspondences, and unrealistic sampling patterns. We evaluate a suite of top methods on a suite of real-world datasets. We show that despite the emphasis placed on learning, most performance gains are caused by pre- and post-processing steps.
  arXiv  Detail & Related papers  (2023-04-04T22:45:50Z)
• Online Hyperparameter Optimization for Class-Incremental Learning [99.70569355681174]
  Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. We propose an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori.
  arXiv  Detail & Related papers  (2023-01-11T17:58:51Z)
• Improving information retention in large scale online continual learning [99.73847522194549]
  Online continual learning aims to adapt efficiently to new data while retaining existing knowledge. Recent work suggests that information retention remains a problem in large scale OCL even when the replay buffer is unlimited. We propose using a moving average family of methods to improve optimization for non-stationary objectives.
  arXiv  Detail & Related papers  (2022-10-12T16:59:43Z)
• Offline Reinforcement Learning with Fisher Divergence Critic Regularization [41.085156836450466]
  We propose an alternative approach to encouraging the learned policy to stay close to the data, namely parameterizing the critic as the log-behavior-policy. Behavior regularization then corresponds to an appropriate regularizer on the offset term. Our algorithm Fisher-BRC achieves both improved performance and faster convergence over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-14T22:11:40Z)
• Generalized Variational Continual Learning [33.194866396158005]
  Two main approaches to continuous learning are Online Elastic Weight Consolidation and Variational Continual Learning. We show that applying this modification to mitigate Online EWC as a limiting case, allowing baselines between the two approaches. In order to the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, mitigate neural networks with task-specific FiLM layers.
  arXiv  Detail & Related papers  (2020-11-24T19:07:39Z)
• Tracking Performance of Online Stochastic Learners [57.14673504239551]
  Online algorithms are popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches. When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy. We establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models.
  arXiv  Detail & Related papers  (2020-04-04T14:16:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.