On the Dynamics & Transferability of Latent Generalization during Memorization

• URL: http://arxiv.org/abs/2603.19865v1
• Date: Fri, 20 Mar 2026 11:30:57 GMT
• Title: On the Dynamics & Transferability of Latent Generalization during Memorization
• Abstract summary: We study the origin and dynamics over training of latent generalization abilities during memorization.<n>We find that latent generalization abilities largely peak early in training, with model generalization.<n>We devise a way to transfer the latent generalization present in last-layer representations to the model using a new linear probe.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep networks have been known to have extraordinary generalization abilities, via mechanisms that aren't yet well understood. It is also known that upon shuffling labels in the training data to varying degrees, deep networks, trained with standard methods, can still achieve perfect or high accuracy on this corrupted training data. This phenomenon is called memorization, and typically comes at the cost of poorer generalization to true labels. Our recent work has demonstrated, that the internal representations of such models retain significantly better latent generalization abilities than is directly apparent from the model. In particular, it has been shown that such latent generalization can be recovered via simple probes (called MASC probes) on the layer-wise representations of the model. However, the origin and dynamics over training of this latent generalization during memorization is not well understood. Here, we track the training dynamics, empirically, and find that latent generalization abilities largely peak early in training, with model generalization. Next, we investigate to what extent the specific nature of the MASC probe is critical for our ability to extract latent generalization from the model's layerwise outputs. To this end, we first examine the mathematical structure of the MASC probe and show that it is a quadratic classifier, i.e. is non-linear. This brings up the question of the extent to which this latent generalization might be linearly decodable from layerwise outputs. To investigate this, we designed a new linear probe for this setting. Next, we consider the question of whether it is possible to transfer latent generalization to model generalization by directly editing model weights. To this end, we devise a way to transfer the latent generalization present in last-layer representations to the model using the new linear probe.

Related papers

• Understanding Generalization in Diffusion Models via Probability Flow Distance [7.675910526644439]
  We introduce probability flow distance ($texttPFD$) to measure distributional generalization.<n>We empirically uncover several key generalization behaviors in diffusion models.
  arXiv  Detail & Related papers  (2025-05-26T15:23:50Z)
• Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models [56.032091696552094]
  Generalization in natural data domains is progressively achieved during training before the onset of memorization.<n>Generalization vs. memorization is then best understood as a competition between time scales.<n>We show that this phenomenology is recovered in diffusion models learning a simple probabilistic context-free grammar with random rules.
  arXiv  Detail & Related papers  (2025-05-22T17:40:08Z)
• Learning Model Successors [31.25792515137003]
  We argue that the unifying theme is induction, based on finite samples observed in training.<n>We propose a novel learning paradigm, Inductive Learning, which involves a central concept called model successors.
  arXiv  Detail & Related papers  (2025-01-31T22:27:09Z)
• Decoding Generalization from Memorization in Deep Neural Networks [0.0]
  Deep Neural Networks that generalize well have been key to the dramatic success of Deep Learning in recent years.<n>It has been known that deep networks possess the ability to memorize training data, as evidenced by perfect or high training accuracies on models trained with corrupted data that have class labels shuffled to varying degrees.<n>Here, we provide evidence for the latter possibility by demonstrating, empirically, that such models possess information in their representations for substantially improved generalization, even in the face of memorization.
  arXiv  Detail & Related papers  (2025-01-24T18:01:27Z)
• Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization [52.16435732772263]
  Second-order optimization has been shown to accelerate the training of deep neural networks in many applications. However, generalization properties of second-order methods are still being debated. We show for the first time that exact Gauss-Newton (GN) updates take on a tractable form in a class of deep architectures.
  arXiv  Detail & Related papers  (2024-11-12T17:58:40Z)
• The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels [8.670973157365689]
  Prior work argued that the implicit bias of structured state space models (SSMs) leads to generalization in a setting where data is generated by a low dimensional teacher.<n>We prove that while implicit bias leads to generalization under many choices of training data, there exist special examples whose inclusion in training completely distorts the implicit bias.<n>This failure occurs despite the special training examples being labeled by the teacher, i.e. having clean labels!
  arXiv  Detail & Related papers  (2024-10-14T13:08:15Z)
• Grokking at the Edge of Linear Separability [1.024113475677323]
  grokking is delayed generalization accompanied by non-monotonic test loss behavior.<n>We find that grokking arises naturally even when the parameters of the problem are close to a critical point.
  arXiv  Detail & Related papers  (2024-10-06T14:08:42Z)
• Theoretical Characterization of the Generalization Performance of Overfitted Meta-Learning [70.52689048213398]
  This paper studies the performance of overfitted meta-learning under a linear regression model with Gaussian features. We find new and interesting properties that do not exist in single-task linear regression. Our analysis suggests that benign overfitting is more significant and easier to observe when the noise and the diversity/fluctuation of the ground truth of each training task are large.
  arXiv  Detail & Related papers  (2023-04-09T20:36:13Z)
• When Neural Networks Fail to Generalize? A Model Sensitivity Perspective [82.36758565781153]
  Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario, namely Single Domain Generalization (Single-DG) We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity" We propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies.
  arXiv  Detail & Related papers  (2022-12-01T20:15:15Z)
• Do Quantum Circuit Born Machines Generalize? [58.720142291102135]
  We present the first work in the literature that presents the QCBM's generalization performance as an integral evaluation metric for quantum generative models. We show that the QCBM is able to effectively learn the reweighted dataset and generate unseen samples with higher quality than those in the training set.
  arXiv  Detail & Related papers  (2022-07-27T17:06:34Z)
• Stochastic Training is Not Necessary for Generalization [57.04880404584737]
  It is widely believed that the implicit regularization of gradient descent (SGD) is fundamental to the impressive generalization behavior we observe in neural networks. In this work, we demonstrate that non-stochastic full-batch training can achieve strong performance on CIFAR-10 that is on-par with SGD.
  arXiv  Detail & Related papers  (2021-09-29T00:50:00Z)
• Generalization by design: Shortcuts to Generalization in Deep Learning [7.751691910877239]
  We show that good generalization may be instigated by bounded spectral products over layers leading to a novel geometric regularizer. Backed up by theory we further demonstrate that "generalization by design" is practically possible and that good generalization may be encoded into the structure of the network.
  arXiv  Detail & Related papers  (2021-07-05T20:01:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.