Mapping Post-Training Forgetting in Language Models at Scale

• URL: http://arxiv.org/abs/2510.17776v1
• Date: Mon, 20 Oct 2025 17:35:47 GMT
• Title: Mapping Post-Training Forgetting in Language Models at Scale
• Authors: Jackson Harmon, Andreas Hochlehnert, Matthias Bethge, Ameya Prabhu,
• Abstract summary: Scaled post-training now drives many of the largest capability gains in language models.<n>We propose a sample-wise paradigm to measure what is forgotten and when backward transfer occurs.<n>Our framework offers a practical yardstick for mapping how post-training alters pretrained knowledge at scale.
• Score: 21.32247361921916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scaled post-training now drives many of the largest capability gains in language models (LMs), yet its effect on pretrained knowledge remains poorly understood. Not all forgetting is equal: Forgetting one fact (e.g., a U.S. president or an API call) does not "average out" by recalling another. Hence, we propose a sample-wise paradigm to measure what is forgotten and when backward transfer occurs. Our metric counts 1->0 transitions (correct before post-training, incorrect after) to quantify forgetting and 0->1 transitions to quantify backward transfer. Traditional task averages conflate these effects and obscure large changes. For multiple-choice benchmarks, we add chance-adjusted variants that subtract the expected contribution of random guessing from pre- and post-training accuracies. We apply this framework across post-training stages, model sizes, and data scales. Our large-scale analysis shows that: (1) Domain-continual pretraining induces moderate forgetting with low-to-moderate backward transfer; (2) RL/SFT post-training applied to base models and Instruction tuning yields moderate-to-large backward transfer on math and logic with overall low-to-moderate forgetting; (3) Applying RL/SFT to instruction-tuned models is sensitive on data scale: at small scales, both forgetting and backward transfer are small; at larger scales, effects are mixed and warrant further study with better controls; (4) Model merging does not reliably mitigate forgetting. Overall, our framework offers a practical yardstick for mapping how post-training alters pretrained knowledge at scale -- enabling progress towards generally capable AI systems.

Related papers

• Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks [17.067788440109137]
  Mixture-of-Experts (MoE) models are now standard in state-of-the-art systems.<n>We investigate how MoE sparsity influences two distinct capability regimes: memorization skills and reasoning skills.
  arXiv  Detail & Related papers  (2025-08-26T04:31:28Z)
• Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models [19.136589266017694]
  Training large language models typically involves pre-training on massive corpora.<n>New data often causes distribution shifts, leading to performance degradation on previously learned tasks.<n>We take a deeper look at two popular proposals for addressing this distribution shift: experience replay and gradient alignment.
  arXiv  Detail & Related papers  (2025-08-03T20:07:15Z)
• Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning [75.31797502976802]
  We evaluate over 20 open-weight reasoning-tuned models across a broad suite of tasks.<n>We find that most models that succeed in math fail to transfer their gains to other domains.<n>Our results suggest a need to rethink standard post-training recipes.
  arXiv  Detail & Related papers  (2025-07-01T05:23:05Z)
• Scaling Laws for Downstream Task Performance in Machine Translation [27.278023091494507]
  We study how the choice of the pretraining data and its size affect downstream performance (translation quality) as judged by metrics such as BLEU and COMET scores.<n>With sufficient alignment, both downstream cross-entropy and translation quality scores improve monotonically with more pretraining data.
  arXiv  Detail & Related papers  (2024-02-06T17:31:20Z)
• An Emulator for Fine-Tuning Large Language Models using Small Language Models [91.02498576056057]
  We introduce emulated fine-tuning (EFT), a principled and practical method for sampling from a distribution that approximates the result of pre-training and fine-tuning at different scales. We show that EFT enables test-time adjustment of competing behavioral traits like helpfulness and harmlessness without additional training. Finally, a special case of emulated fine-tuning, which we call LM up-scaling, avoids resource-intensive fine-tuning of large pre-trained models by ensembling them with small fine-tuned models.
  arXiv  Detail & Related papers  (2023-10-19T17:57:16Z)
• Inverse Scaling: When Bigger Isn't Better [80.42834197416444]
  Large language models (LMs) show predictable improvements to overall loss with increased scale. We present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale.
  arXiv  Detail & Related papers  (2023-06-15T20:11:23Z)
• Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model [89.8764435351222]
  We propose a new family of unbiased estimators called WTA-CRS, for matrix production with reduced variance. Our work provides both theoretical and experimental evidence that, in the context of tuning transformers, our proposed estimators exhibit lower variance compared to existing ones.
  arXiv  Detail & Related papers  (2023-05-24T15:52:08Z)
• Scaling Laws Beyond Backpropagation [64.0476282000118]
  We study the ability of Direct Feedback Alignment to train causal decoder-only Transformers efficiently. We find that DFA fails to offer more efficient scaling than backpropagation.
  arXiv  Detail & Related papers  (2022-10-26T10:09:14Z)
• Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models [46.24479693469042]
  This paper shows that 1) pre-training loss cannot fully explain downstream performance and 2) flatness of the model is well-correlated with downstream performance where pre-training loss is not.
  arXiv  Detail & Related papers  (2022-10-25T17:45:36Z)
• LogME: Practical Assessment of Pre-trained Models for Transfer Learning [80.24059713295165]
  The Logarithm of Maximum Evidence (LogME) can be used to assess pre-trained models for transfer learning. Compared to brute-force fine-tuning, LogME brings over $3000times$ speedup in wall-clock time.
  arXiv  Detail & Related papers  (2021-02-22T13:58:11Z)
• Scaling Laws for Transfer [0.5432984841650929]
  We study scaling laws for transfer learning between distributions in an unsupervised, fine-tuning setting. We find that the effective data transferred is described well in the low data regime by a power-law of parameter count and fine-tuning dataset size.
  arXiv  Detail & Related papers  (2021-02-02T04:07:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.