Continual Learning With Quasi-Newton Methods

• URL: http://arxiv.org/abs/2503.19939v1
• Date: Tue, 25 Mar 2025 07:45:59 GMT
• Title: Continual Learning With Quasi-Newton Methods
• Authors: Steven Vander Eeckt, Hugo Van hamme,
• Abstract summary: Catastrophic forgetting remains a major challenge when neural networks learn tasks sequentially.<n>EWC attempts to address this problem by introducing a Bayesian-inspired regularization loss to preserve knowledge of previously learned tasks.<n>EWC relies on a Laplace approximation where the Hessian is simplified to the diagonal of the Fisher information matrix, assuming uncorrelated model parameters.<n>We introduce Continual Learning with Sampled Quasi-Newton (CSQN), which leverages Quasi-Newton methods to compute more accurate Hessian approximations.
• Score: 12.55972766570669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Catastrophic forgetting remains a major challenge when neural networks learn tasks sequentially. Elastic Weight Consolidation (EWC) attempts to address this problem by introducing a Bayesian-inspired regularization loss to preserve knowledge of previously learned tasks. However, EWC relies on a Laplace approximation where the Hessian is simplified to the diagonal of the Fisher information matrix, assuming uncorrelated model parameters. This overly simplistic assumption often leads to poor Hessian estimates, limiting its effectiveness. To overcome this limitation, we introduce Continual Learning with Sampled Quasi-Newton (CSQN), which leverages Quasi-Newton methods to compute more accurate Hessian approximations. CSQN captures parameter interactions beyond the diagonal without requiring architecture-specific modifications, making it applicable across diverse tasks and architectures. Experimental results across four benchmarks demonstrate that CSQN consistently outperforms EWC and other state-of-the-art baselines, including rehearsal-based methods. CSQN reduces EWC's forgetting by 50 percent and improves its performance by 8 percent on average. Notably, CSQN achieves superior results on three out of four benchmarks, including the most challenging scenarios, highlighting its potential as a robust solution for continual learning.

Related papers

• A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU [7.030422837091069]
  Post-Training Quantization (PTQ) is crucial for efficient model deployment on Ascend NPU.<n>This paper presents a case study of PTQ baselines applied to reasoning-oriented models such as DeepSeek-R1-Distill-Qwen series (1.5B/7B/14B) and QwQ-32B.<n>We evaluate four distinct algorithms, including AWQ, GPTQ, SmoothQuant, and FlatQuant, to cover the spectrum from weight-only compression to advanced rotation-based methods.
  arXiv  Detail & Related papers  (2026-02-06T09:22:09Z)
• Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach [10.914558012458423]
  Deep equilibrium models (DEQs) achieve infinitely deep network representations without stacking layers by exploring fixed points of layer transformations in neural networks.<n>DEQs face the challenge of requiring vastly more computational time for training and inference than conventional methods.<n>This study explored an approach to improve fixed-point convergence and consequently reduce computational time.
  arXiv  Detail & Related papers  (2026-02-03T09:22:56Z)
• What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study [59.44848132298657]
  Post-training quantization (PTQ) usually comes with the cost of large accuracy drops, especially for reasoning tasks under low-bit settings.<n>In this study, we present a systematic empirical study of quantization-aware training (QAT) for reasoning models.
  arXiv  Detail & Related papers  (2026-01-21T11:22:29Z)
• Sat-EnQ: Satisficing Ensembles of Weak Q-Learners for Reliable and Compute-Efficient Reinforcement Learning [0.0]
  We introduce Sat-EnQ, a framework that learns to be good enough'' before optimizing aggressively.<n>In Phase 1, we train an ensemble of lightweight Q-networks under a satisficing objective that limits early value growth.<n>In Phase 2, the ensemble is distilled into a larger network and fine-tuned with standard Double DQN.
  arXiv  Detail & Related papers  (2025-12-28T12:41:09Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Deep Hierarchical Learning with Nested Subspace Networks [53.71337604556311]
  We propose Nested Subspace Networks (NSNs) for large neural networks.<n>NSNs enable a single model to be dynamically and granularly adjusted across a continuous spectrum of compute budgets.<n>We show that NSNs can be surgically applied to pre-trained LLMs and unlock a smooth and predictable compute-performance frontier.
  arXiv  Detail & Related papers  (2025-09-22T15:13:14Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture. Non-smooth regularization is often incorporated into machine learning tasks. We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Large Language Models Can Help Mitigate Barren Plateaus [2.384873896423002]
  Quantum Neural Networks (QNNs) have emerged as a promising approach for various applications, yet their training is often hindered by barren plateaus (BPs)<n>We propose a new Large Language Model (LLM)-driven search framework, AdaInit, that iteratively searches for optimal initial parameters of QNNs to maximize gradient variance and therefore mitigate BPs.
  arXiv  Detail & Related papers  (2025-02-17T05:57:15Z)
• GAQAT: gradient-adaptive quantization-aware training for domain generalization [54.31450550793485]
  We propose a novel Gradient-Adaptive Quantization-Aware Training (GAQAT) framework for DG.<n>Our approach begins by identifying the scale-gradient conflict problem in low-precision quantization.<n>Extensive experiments validate the effectiveness of the proposed GAQAT framework.
  arXiv  Detail & Related papers  (2024-12-07T06:07:21Z)
• Variance-Reduced Cascade Q-learning: Algorithms and Sample Complexity [3.4376560669160394]
  We introduce and analyze a novel model-free algorithm called Variance-Reduced Cascade Q-learning (VRCQ) VRCQ provides superior guarantees in the $ell_infty$-norm compared with the existing model-free approximation-type algorithms.
  arXiv  Detail & Related papers  (2024-08-13T00:34:33Z)
• Sequential Hamiltonian Assembly: Enhancing the training of combinatorial optimization problems on quantum computers [4.385485960663339]
  A central challenge in quantum machine learning is the design and training of parameterized quantum circuits (PQCs) Much like in deep learning, vanishing gradients pose significant obstacles to the trainability of PQCs, arising from various sources. We propose Sequential Hamiltonian Assembly (SHA) to address this issue and facilitate parameter training for quantum applications using global loss functions.
  arXiv  Detail & Related papers  (2024-08-08T20:32:18Z)
• Strategically Conservative Q-Learning [89.17906766703763]
  offline reinforcement learning (RL) is a compelling paradigm to extend RL's practical utility. The major difficulty in offline RL is mitigating the impact of approximation errors when encountering out-of-distribution (OOD) actions. We propose a novel framework called Strategically Conservative Q-Learning (SCQ) that distinguishes between OOD data that is easy and hard to estimate.
  arXiv  Detail & Related papers  (2024-06-06T22:09:46Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Hessian Aware Low-Rank Perturbation for Order-Robust Continual Learning [19.850893012601638]
  Continual learning aims to learn a series of tasks sequentially without forgetting the knowledge acquired from the previous ones. We propose the Hessian Aware Low-Rank Perturbation algorithm for continual learning.
  arXiv  Detail & Related papers  (2023-11-26T01:44:01Z)
• On the efficiency of Stochastic Quasi-Newton Methods for Deep Learning [0.0]
  We study the behaviour of quasi-Newton training algorithms for deep memory networks. We show that quasi-Newtons are efficient and able to outperform in some instances the well-known first-order Adam run.
  arXiv  Detail & Related papers  (2022-05-18T20:53:58Z)
• Training Quantised Neural Networks with STE Variants: the Additive Noise Annealing Algorithm [16.340620299847384]
  Training quantised neural networks (QNNs) is a non-differentiable problem since weights and features are output by piecewise constant functions. The standard solution is to apply the straight-through estimator (STE), using different functions during the inference and computation steps. Several STE variants have been proposed in the literature aiming to maximise the task accuracy of the trained network.
  arXiv  Detail & Related papers  (2022-03-21T20:14:27Z)
• Compact representations of convolutional neural networks via weight pruning and quantization [63.417651529192014]
  We propose a novel storage format for convolutional neural networks (CNNs) based on source coding and leveraging both weight pruning and quantization. We achieve a reduction of space occupancy up to 0.6% on fully connected layers and 5.44% on the whole network, while performing at least as competitive as the baseline.
  arXiv  Detail & Related papers  (2021-08-28T20:39:54Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)
• Cross Learning in Deep Q-Networks [82.20059754270302]
  We propose a novel cross Q-learning algorithm, aim at alleviating the well-known overestimation problem in value-based reinforcement learning methods. Our algorithm builds on double Q-learning, by maintaining a set of parallel models and estimate the Q-value based on a randomly selected network.
  arXiv  Detail & Related papers  (2020-09-29T04:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.