Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning

• URL: http://arxiv.org/abs/2510.11372v1
• Date: Mon, 13 Oct 2025 13:12:46 GMT
• Title: Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning
• Authors: Dean L. Slack, Noura Al Moubayed,
• Abstract summary: We fine-tune Pythia, Llama3, and Mistral models spanning 1.4B-70B parameters on common evaluation datasets.<n>We find that memorisation increases dramatically in the first few epochs, often significantly before either validation perplexity or evaluation performance is optimised.<n>We introduce an n-gram-aware loss regulariser and show that it reduces memorisation across all model families tested by up to 40%.
• Score: 4.145513103869504
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although large language models excel across many tasks, they can memorise training data and thereby expose private or copyrighted text. Most defences target the pre-training stage, leaving memorisation during fine-tuning, especially for domain adaptation and instruction tuning, poorly understood. We fine-tune Pythia, Llama3, and Mistral models spanning 1.4B-70B parameters on common evaluation datasets and track verbatim memorisation throughout training. We find that memorisation increases dramatically in the first few epochs, often significantly before either validation perplexity or evaluation performance is optimised. We use a simple but effective n-gram memorisation score which reliably precedes verbatim memorisation; using it as an early-stopping criterion mitigates memorisation with minimal performance loss. Further, we introduce an n-gram-aware loss regulariser and show that it reduces memorisation across all model families tested by up to 40% while minimising evaluation performance trade-offs when compared to an existing memorisation mitigation strategy. These results yield practical, scalable insights into memorisation dynamics during language model fine-tuning.

Related papers

• Continual Learning via Sparse Memory Finetuning [58.163704181154834]
  We investigate whether sparse parameter updates can enable learning without catastrophic forgetting.<n>By updating only the memory slots that are highly activated by a new piece of knowledge relative to usage on pretraining data, we reduce interference between new knowledge and the model's existing capabilities.<n>We find that sparse memory finetuning learns new knowledge while exhibiting substantially less forgetting.
  arXiv  Detail & Related papers  (2025-10-16T19:44:38Z)
• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning [64.93848182403116]
  Current deep-learning memory models struggle in reinforcement learning environments that are partially observable and long-term. We introduce the Stable Hadamard Memory, a novel memory model for reinforcement learning agents. Our approach significantly outperforms state-of-the-art memory-based methods on challenging partially observable benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T03:50:17Z)
• EMP: Enhance Memory in Data Pruning [18.535687216213628]
  Recently, large language and vision models have shown strong performance, but due to high pre-training and fine-tuning costs, research has shifted towards faster training via dataset pruning. Previous methods used sample loss as an evaluation criterion, aiming to select the most "difficult" samples for training. We propose Enhance Memory Pruning (EMP), which addresses the issue of insufficient memory under high pruning rates by enhancing the model's memory of data, thereby improving its performance.
  arXiv  Detail & Related papers  (2024-08-28T10:29:52Z)
• Causal Estimation of Memorisation Profiles [58.20086589761273]
  Understanding memorisation in language models has practical and societal implications. Memorisation is the causal effect of training with an instance on the model's ability to predict that instance. This paper proposes a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics.
  arXiv  Detail & Related papers  (2024-06-06T17:59:09Z)
• Unintended Memorization in Large ASR Models, and How to Mitigate It [16.047859326721046]
  auditing memorization in large non-auto-regressive automatic speech recognition (ASR) models has been challenging. We design a simple auditing method to measure memorization in large ASR models without the extra compute overhead. We show that in large-scale distributed training, clipping the average gradient on each compute core maintains neutral model quality and compute cost.
  arXiv  Detail & Related papers  (2023-10-18T06:45:49Z)
• AdaLomo: Low-memory Optimization with Adaptive Learning Rate [59.64965955386855]
  We introduce low-memory optimization with adaptive learning rate (AdaLomo) for large language models. AdaLomo results on par with AdamW, while significantly reducing memory requirements, thereby lowering the hardware barrier to training large language models.
  arXiv  Detail & Related papers  (2023-10-16T09:04:28Z)
• DPZero: Private Fine-Tuning of Language Models without Backpropagation [49.365749361283704]
  We introduce DPZero, a novel private zeroth-order algorithm with nearly dimension-independent rates. The memory efficiency of DPZero is demonstrated in privately fine-tuning RoBERTa and OPT on several downstream tasks.
  arXiv  Detail & Related papers  (2023-10-14T18:42:56Z)
• What do larger image classifiers memorise? [64.01325988398838]
  We show that training examples exhibit an unexpectedly diverse set of memorisation trajectories across model sizes. We find that knowledge distillation, an effective and popular model compression technique, tends to inhibit memorisation, while also improving generalisation.
  arXiv  Detail & Related papers  (2023-10-09T01:52:07Z)
• Emergent and Predictable Memorization in Large Language Models [23.567027014457775]
  Memorization, or the tendency of large language models to output entire sequences from their training data verbatim, is a key concern for safely deploying language models. We seek to predict which sequences will be memorized before a large model's full train-time by extrapolating the memorization behavior of lower-compute trial runs. We provide further novel discoveries on the distribution of memorization scores across models and data.
  arXiv  Detail & Related papers  (2023-04-21T17:58:31Z)
• An Evaluation of Memory Optimization Methods for Training Neural Networks [12.534553433992606]
  Development of memory optimization methods (MOMs) has emerged as a solution to address the memory bottleneck encountered when training large models. To examine the practical value of various MOMs, we have conducted a thorough analysis of existing literature from a systems perspective. Our analysis has revealed a notable challenge within the research community: the absence of standardized metrics for effectively evaluating the efficacy of MOMs.
  arXiv  Detail & Related papers  (2023-03-26T05:40:35Z)
• Saliency-Augmented Memory Completion for Continual Learning [8.243137410556495]
  How to forget is a problem continual learning must address. Our paper proposes a new saliency-augmented memory completion framework for continual learning.
  arXiv  Detail & Related papers  (2022-12-26T18:06:39Z)
• A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning [56.450090618578]
  Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. We show that when counting the model size into the total budget and comparing methods with aligned memory size, saving models do not consistently work. We propose a simple yet effective baseline, denoted as MEMO for Memory-efficient Expandable MOdel.
  arXiv  Detail & Related papers  (2022-05-26T08:24:01Z)
• Sequential memory improves sample and memory efficiency in Episodic Control [0.0]
  State of the art deep reinforcement learning algorithms are sample inefficient due to the large number of episodes they require to achieve performance. ERL algorithms, inspired by the mammalian hippocampus, typically use extended memory systems to bootstrap learning from past events to overcome this sample-inefficiency problem. Here, we demonstrate that including a bias in the acquired memory content derived from the order of episodic sampling improves both the sample and memory efficiency of an episodic control algorithm.
  arXiv  Detail & Related papers  (2021-12-29T18:42:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.