Convergence and Divergence of Language Models under Different Random Seeds

• URL: http://arxiv.org/abs/2509.26643v1
• Date: Tue, 30 Sep 2025 17:59:50 GMT
• Title: Convergence and Divergence of Language Models under Different Random Seeds
• Abstract summary: We measure convergence as the expected per-token Kullback--Leibler divergence across seeds.<n>We observe that larger models reconverge faster in later training stages, while smaller models never actually reconverge.
• Score: 26.31262670498101
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we investigate the convergence of language models (LMs) trained under different random seeds, measuring convergence as the expected per-token Kullback--Leibler (KL) divergence across seeds. By comparing LM convergence as a function of model size and training checkpoint, we identify a four-phase convergence pattern: (i) an initial uniform phase, (ii) a sharp-convergence phase, (iii) a sharp-divergence phase, and (iv) a slow-reconvergence phase. Further, we observe that larger models reconverge faster in later training stages, while smaller models never actually reconverge; these results suggest that a certain model size may be necessary to learn stable distributions. Restricting our analysis to specific token frequencies or part-of-speech (PoS) tags further reveals that convergence is uneven across linguistic categories: frequent tokens and function words converge faster and more reliably than their counterparts (infrequent tokens and content words). Overall, our findings highlight factors that influence the stability of the learned distributions in model training.

Related papers

• Memorisation, convergence and generalisation in generative models [13.652405522895128]
  We show that generalisation in generative models decomposes into at least two distinct objectives.<n>These objectives correspond to two different distances between true and learnt data distribution.
  arXiv  Detail & Related papers  (2026-05-20T17:00:37Z)
• UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
  Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
  arXiv  Detail & Related papers  (2026-02-12T18:59:49Z)
• Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models [50.99097734404912]
  We show that RL-friendly models exhibit intra-class compactness and inter-class separation in their probability assignments to correct vs. incorrect responses.<n>Experiments across six mathematical benchmarks show consistent improvements across all model families, with gains up to 5.9 points on AIME24.
  arXiv  Detail & Related papers  (2026-01-11T13:34:44Z)
• A Convergence Theory for Diffusion Language Models: An Information-Theoretic Perspective [8.15094483029656]
  Diffusion models enable parallel token sampling, leading to faster generation and eliminating left-to-right generation constraints.<n>We develop convergence guarantees for diffusion language models from an information-theoretic perspective.<n>These results offer novel theoretical insights into the practical effectiveness of diffusion language models.
  arXiv  Detail & Related papers  (2025-05-27T16:24:20Z)
• Multi-Level Collaboration in Model Merging [56.31088116526825]
  This paper explores the intrinsic connections between model merging and model ensembling.<n>We find that even when previous restrictions are not met, there is still a way for model merging to attain a near-identical and superior performance similar to that of ensembling.
  arXiv  Detail & Related papers  (2025-03-03T07:45:04Z)
• Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts [1.9458156037869137]
  convergent learning is the degree to which neural systems arrive at similar internal representations.<n>We present a large-scale audit of convergent learning spanning dozens of vision models and thousands of layer-pair comparisons.<n>Findings fill critical gaps in our understanding of representational convergence, with implications for neuroscience and AI.
  arXiv  Detail & Related papers  (2025-02-26T00:04:24Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Training Trajectories of Language Models Across Scales [99.38721327771208]
  Scaling up language models has led to unprecedented performance gains. How do language models of different sizes learn during pre-training? Why do larger language models demonstrate more desirable behaviors?
  arXiv  Detail & Related papers  (2022-12-19T19:16:29Z)
• Pairwise Learning via Stagewise Training in Proximal Setting [0.0]
  We combine adaptive sample size and importance sampling techniques for pairwise learning, with convergence guarantees for nonsmooth convex pairwise loss functions. We demonstrate that sampling opposite instances at each reduces the variance of the gradient, hence accelerating convergence.
  arXiv  Detail & Related papers  (2022-08-08T11:51:01Z)
• A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta [14.857119814202754]
  Mini-batch SGD with momentum is a fundamental algorithm for learning large predictive models. We develop a new analytic framework to analyze mini-batch SGD for linear models at different momenta and sizes of batches.
  arXiv  Detail & Related papers  (2022-06-22T14:15:35Z)
• Multi-timescale Representation Learning in LSTM Language Models [69.98840820213937]
  Language models must capture statistical dependencies between words at timescales ranging from very short to very long. We derived a theory for how the memory gating mechanism in long short-term memory language models can capture power law decay. Experiments showed that LSTM language models trained on natural English text learn to approximate this theoretical distribution.
  arXiv  Detail & Related papers  (2020-09-27T02:13:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.