MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

• URL: http://arxiv.org/abs/2603.09892v1
• Date: Tue, 10 Mar 2026 16:49:44 GMT
• Title: MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
• Abstract summary: We propose an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting.<n>Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines.
• Score: 36.119948355819
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.

Related papers

• Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training [11.074386515074446]
  "Memorization-guided Data Reuse" is a training paradigm that adaptively determines when and how data should be reused.<n>Preliminary experiments reveal a consistent memorization-driven regime.
  arXiv  Detail & Related papers  (2026-07-06T11:57:32Z)
• MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models [64.2820121997882]
  MIITA is a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage.<n>It stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues.<n>Experiments show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.
  arXiv  Detail & Related papers  (2026-05-20T03:03:04Z)
• Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers [7.6344611819427035]
  Multiple Embedding Replay Selection, MERS, replaces the buffer selection module with a graph-based approach.<n>We show consistent improvements over SOTA selection strategies across a range of continual learning algorithms.<n>MERS is a practical, drop-in enhancement for replay-based continual learning.
  arXiv  Detail & Related papers  (2026-04-09T15:09:02Z)
• MemDLM: Memory-Enhanced DLM Training [56.40248490616793]
  Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models.<n>They suffer from a notable train-inference mismatch.<n>We propose MemDLM (Memory-Enhanced DLM), which embeds a simulated denoising process into training.
  arXiv  Detail & Related papers  (2026-03-23T17:39:56Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning [63.20028888397869]
  FOREVER (FORgEtting curVe-inspired mEmory) is a novel framework that aligns replay schedules with a model-centric notion of time.<n>Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay.
  arXiv  Detail & Related papers  (2026-01-07T13:55:14Z)
• Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning [51.07663354001582]
  Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task.<n>We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches.<n>We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods.
  arXiv  Detail & Related papers  (2025-12-01T15:56:00Z)
• R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models [15.189716355297326]
  Replay Tuning (R-Tuning) is a novel framework designed for the continual adaptation of pre-trained time-series models.<n>R-Tuning constructs a unified latent space that captures both prior and current task knowledge through a frequency-aware replay strategy.
  arXiv  Detail & Related papers  (2025-11-12T08:32:08Z)
• MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment [14.392166280035122]
  Large language models (LLMs) deployed on edge servers are increasingly used in latency-sensitive applications such as personalized assistants, recommendation, and content moderation.<n>Existing retraining strategies either delay model updates, over-commit resources to retraining, or overlook iteration-level retraining granularity.<n>We propose MACE, a hybrid LLM system that colocates concurrent inference (prefill, decode) and fine-tuning, with intelligent memory management to maximize task performance while promising inference throughput.
  arXiv  Detail & Related papers  (2025-09-28T18:45:28Z)
• Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs [76.40876036912537]
  Large Language Models (LLMs) demonstrate strong few-shot adaptability without requiring fine-tuning.<n>Current Visual Foundation Models (VFMs) require explicit fine-tuning with sufficient tuning data.<n>We propose a framework, LoRA Recycle, that distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective.
  arXiv  Detail & Related papers  (2024-12-03T07:25:30Z)
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization [165.98557106089777]
  A key challenge is to enhance the capabilities of large language models (LLMs) amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization.
  arXiv  Detail & Related papers  (2024-02-22T04:10:57Z)
• Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization [24.143811670210546]
  Generative experience replay addresses the challenge of learning new tasks accumulatively without forgetting. We introduce a time-aware regularization method to fine-tune the three training objective terms used for generative replay. Experimental results indicate that our method pushes the limit of brain-inspired continual learners under such strict settings.
  arXiv  Detail & Related papers  (2023-10-05T21:07:45Z)
• AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning [16.457330925212606]
  We present adaptive-experience replay (AdaER) to address the challenge of continual lifelong learning. AdaER consists of two stages: memory replay and memory update. Results: AdaER outperforms existing continual lifelong learning baselines.
  arXiv  Detail & Related papers  (2023-08-07T01:25:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.