Continual Learning via Sparse Memory Finetuning

• URL: http://arxiv.org/abs/2510.15103v1
• Date: Thu, 16 Oct 2025 19:44:38 GMT
• Title: Continual Learning via Sparse Memory Finetuning
• Authors: Jessy Lin, Luke Zettlemoyer, Gargi Ghosh, Wen-Tau Yih, Aram Markosyan, Vincent-Pierre Berges, Barlas Oğuz,
• Abstract summary: We investigate whether sparse parameter updates can enable learning without catastrophic forgetting.<n>By updating only the memory slots that are highly activated by a new piece of knowledge relative to usage on pretraining data, we reduce interference between new knowledge and the model's existing capabilities.<n>We find that sparse memory finetuning learns new knowledge while exhibiting substantially less forgetting.
• Score: 58.163704181154834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern language models are powerful, but typically static after deployment. A major obstacle to building models that continually learn over time is catastrophic forgetting, where updating on new data erases previously acquired capabilities. Motivated by the intuition that mitigating forgetting is challenging because trainable parameters are shared across all tasks, we investigate whether sparse parameter updates can enable learning without catastrophic forgetting. We introduce sparse memory finetuning, leveraging memory layer models (Berges et al., 2024), which are sparsely updated by design. By updating only the memory slots that are highly activated by a new piece of knowledge relative to usage on pretraining data, we reduce interference between new knowledge and the model's existing capabilities. We evaluate learning and forgetting compared to full finetuning and parameter-efficient finetuning with LoRA on two question answering tasks. We find that sparse memory finetuning learns new knowledge while exhibiting substantially less forgetting: while NaturalQuestions F1 drops by 89% after full finetuning on new facts and 71% with LoRA, sparse memory finetuning yields only an 11% drop with the same level of new knowledge acquisition. Our results suggest sparsity in memory layers offers a promising path toward continual learning in large language models.

Related papers

• Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning [4.145513103869504]
  We fine-tune Pythia, Llama3, and Mistral models spanning 1.4B-70B parameters on common evaluation datasets.<n>We find that memorisation increases dramatically in the first few epochs, often significantly before either validation perplexity or evaluation performance is optimised.<n>We introduce an n-gram-aware loss regulariser and show that it reduces memorisation across all model families tested by up to 40%.
  arXiv  Detail & Related papers  (2025-10-13T13:12:46Z)
• Pretraining with hierarchical memories: separating long-tail and common knowledge [32.22296691842835]
  We introduce small language models that access large hierarchical parametric memory banks encoding world knowledge.<n>During pretraining and inference, we fetch a small, context-dependent memory block and add it to the model.<n>Our pretraining learns to store long-tail world knowledge in the memory parameters, while the small language model acts as an anchor capturing general reasoning abilities.
  arXiv  Detail & Related papers  (2025-09-29T17:59:50Z)
• Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Models [51.20499954955646]
  Large language models (LLMs) acquire vast amounts of knowledge from extensive text corpora during the pretraining phase. In later stages such as fine-tuning and inference, the model may encounter knowledge not covered in the initial training. We propose a two-stage fine-tuning strategy to improve the model's overall test accuracy and knowledge retention.
  arXiv  Detail & Related papers  (2024-10-08T08:35:16Z)
• TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning [7.627299398469962]
  We introduce TEAL, a novel approach to populate the memory with exemplars.<n>We show that TEAL enhances the average accuracy of existing class-incremental methods.
  arXiv  Detail & Related papers  (2024-06-30T12:09:08Z)
• R-Tuning: Instructing Large Language Models to Say `I Don't Know' [66.11375475253007]
  Large language models (LLMs) have revolutionized numerous domains with their impressive performance but still face their challenges. Previous instruction tuning methods force the model to complete a sentence no matter whether the model knows the knowledge or not. We present a new approach called Refusal-Aware Instruction Tuning (R-Tuning) Experimental results demonstrate R-Tuning effectively improves a model's ability to answer known questions and refrain from answering unknown questions.
  arXiv  Detail & Related papers  (2023-11-16T08:45:44Z)
• Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models [53.52344131257681]
  We propose a new paradigm for fine-tuning called F-Learning, which employs parametric arithmetic to facilitate the forgetting of old knowledge and learning of new knowledge. Experimental results on two publicly available datasets demonstrate that our proposed F-Learning can obviously improve the knowledge updating performance of both full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2023-11-14T09:12:40Z)
• Saliency-Augmented Memory Completion for Continual Learning [8.243137410556495]
  How to forget is a problem continual learning must address. Our paper proposes a new saliency-augmented memory completion framework for continual learning.
  arXiv  Detail & Related papers  (2022-12-26T18:06:39Z)
• A Memory Transformer Network for Incremental Learning [64.0410375349852]
  We study class-incremental learning, a training setup in which new classes of data are observed over time for the model to learn from. Despite the straightforward problem formulation, the naive application of classification models to class-incremental learning results in the "catastrophic forgetting" of previously seen classes. One of the most successful existing methods has been the use of a memory of exemplars, which overcomes the issue of catastrophic forgetting by saving a subset of past data into a memory bank and utilizing it to prevent forgetting when training future tasks.
  arXiv  Detail & Related papers  (2022-10-10T08:27:28Z)
• Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship [54.73817402934303]
  We propose Experience Continual Replay (ERR), a bi-level learning framework to adaptively tune task-wise to achieve a better stability plasticity' tradeoff. ERR can consistently improve the performance of all baselines and surpass current state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-31T12:05:22Z)
• Representation Memorization for Fast Learning New Knowledge without Forgetting [36.55736909586313]
  The ability to quickly learn new knowledge is a big step towards human-level intelligence. We consider scenarios that require learning new classes or data distributions quickly and incrementally over time. We propose "Memory-based Hebbian Adaptation" to tackle the two major challenges.
  arXiv  Detail & Related papers  (2021-08-28T07:54:53Z)
• Continual Learning via Bit-Level Information Preserving [88.32450740325005]
  We study the continual learning process through the lens of information theory. We propose Bit-Level Information Preserving (BLIP) that preserves the information gain on model parameters. BLIP achieves close to zero forgetting while only requiring constant memory overheads throughout continual learning.
  arXiv  Detail & Related papers  (2021-05-10T15:09:01Z)
• ZS-IL: Looking Back on Learned ExperiencesFor Zero-Shot Incremental Learning [9.530976792843495]
  We propose an on-call transfer set to provide past experiences whenever a new class arises in the data stream. ZS-IL demonstrates significantly better performance on the well-known datasets (CIFAR-10, Tiny-ImageNet) in both Task-IL and Class-IL settings.
  arXiv  Detail & Related papers  (2021-03-22T22:43:20Z)
• iTAML: An Incremental Task-Agnostic Meta-learning Approach [123.10294801296926]
  Humans can continuously learn new knowledge as their experience grows. Previous learning in deep neural networks can quickly fade out when they are trained on a new task. We introduce a novel meta-learning approach that seeks to maintain an equilibrium between all encountered tasks.
  arXiv  Detail & Related papers  (2020-03-25T21:42:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.