Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers

• URL: http://arxiv.org/abs/2604.11508v2
• Date: Thu, 16 Apr 2026 06:09:42 GMT
• Title: Not All Forgetting Is Equal: Architecture-Dependent Retention Dynamics in Fine-Tuned Image Classifiers
• Abstract summary: We track per-sample correctness at every intrinsic during fine-tuning of ResNet-18 and DeiT-Small.<n>Fifth, a sample's loss after head warmup predicts its long-term decay constant.<n>Findings suggest that architectural diversity in ensembles provides retention coverage.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning pretrained image classifiers is standard practice, yet which individual samples are forgotten during this process, and whether forgetting patterns are stable or architecture dependent, remains unclear. Understanding these dynamics has direct implications for curriculum design, data pruning, and ensemble construction. We track per-sample correctness at every epoch during fine-tuning of ResNet-18 and DeiT-Small on a retinal OCT dataset (7 classes, 56:1 imbalance) and CUB-200-2011 (200 bird species), fitting Ebbinghaus-style exponential decay curves to each sample's retention trace. Five findings emerge. First, the two architectures forget fundamentally different samples: Jaccard overlap of the top 10 percent most-forgotten is 0.34 on OCTDL and 0.15 on CUB-200. Second, ViT forgetting is more structured (mean $R^2 = 0.74$) than CNN forgetting ($R^2 = 0.52$). Third, per-sample forgetting is stochastic across random seeds (Spearman $ρ\approx 0.01$), challenging the assumption that sample difficulty is an intrinsic property. Fourth, class-level forgetting is consistent and semantically interpretable: visually similar species are forgotten most, distinctive ones least. Fifth, a sample's loss after head warmup predicts its long-term decay constant ($ρ= 0.30$ to $0.50$, $p < 10^{-45}$). These findings suggest that architectural diversity in ensembles provides complementary retention coverage, and that curriculum or pruning methods based on per-sample difficulty may not generalize across runs. A spaced repetition sampler built on these decay constants does not outperform random sampling, indicating that static scheduling cannot exploit unstable per-sample signals.

Related papers

• GRACE: Gated Refinement for Accurate Causal Edge Discovery in High-Dimensional Time Series [0.0]
  Fast linear CI skeleton provides high-recall candidates.<n>A single gated model prunes false positives by learning which edges genuinely improve prediction.<n> GRACE matches or exceeds expensive nonlinear CI tests at a fraction of the cost.
  arXiv  Detail & Related papers  (2026-06-22T19:26:38Z)
• Rethinking Incompleteness: Formalizing Protocol Divergence and Train-Once Learning for Robust IMVC [13.498020856656494]
  IMVC evaluation retrains separate models for different missing-data configurations.<n>We show that this paradigm obscures a fundamental vulnerability: missing rate alone is insufficient to characterize data incompleteness.<n>We formalize this phenomenon as incompleteness divergence, providing measures that capture structural disparities across missing-data protocols.
  arXiv  Detail & Related papers  (2026-06-03T13:24:09Z)
• Stochastic Attention via Langevin Dynamics on the Modern Hopfield Energy [0.0]
  We show that Langevin sampling from the corresponding distribution yields attention: a training-free sampler controlled by a single temperature.<n>Because the energy gradient equals the attention map, no score network, training loop, or learned model is required.
  arXiv  Detail & Related papers  (2026-03-06T20:50:30Z)
• Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning [61.07540493350384]
  Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth and the teacher's own predictions.<n>We show that for any prediction risk, the optimally mixed student improves upon the ridge teacher for every regularization level.<n>We propose a consistent one-shot tuning method to estimate $star$ without grid search, sample splitting, or refitting.
  arXiv  Detail & Related papers  (2026-02-19T17:21:15Z)
• Information Hidden in Gradients of Regression with Target Noise [2.8911861322232686]
  We show that the gradients alone can reveal the Hessian.<n>We provide non-asymptotic operator-norm guarantees under sub-Gaussian inputs.
  arXiv  Detail & Related papers  (2026-01-26T14:50:16Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• From Continual Learning to SGD and Back: Better Rates for Continual Linear Models [50.11453013647086]
  We analyze the forgetting, i.e., loss on previously seen tasks, after $k$ iterations.<n>We develop novel last-iterate upper bounds in the realizable least squares setup.<n>We prove for the first time that randomization alone, with no task repetition, can prevent catastrophic in sufficiently long task sequences.
  arXiv  Detail & Related papers  (2025-04-06T18:39:45Z)
• Distributionally Robust Classification on a Data Budget [26.69877485937123]
  We show that standard ResNet-50 trained with the cross-entropy loss on 2.4 million image samples can attain comparable robustness to a CLIP ResNet-50 trained on 400 million samples. This is the first result showing (near) state-of-the-art distributional robustness on limited data budgets.
  arXiv  Detail & Related papers  (2023-08-07T15:30:02Z)
• Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models [49.81937966106691]
  We develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach.
  arXiv  Detail & Related papers  (2023-06-15T16:30:08Z)
• Deep learning for $\psi$-weakly dependent processes [0.0]
  We perform deep neural networks for learning $psi$-weakly dependent processes. The consistency of the empirical risk minimization algorithm in the class of deep neural networks predictors is established. Some simulation results are provided, as well as an application to the US recession data.
  arXiv  Detail & Related papers  (2023-02-01T09:31:15Z)
• Characterizing Datapoints via Second-Split Forgetting [93.99363547536392]
  We propose $$-second-$split$ $forgetting$ $time$ (SSFT), a complementary metric that tracks the epoch (if any) after which an original training example is forgotten. We demonstrate that $mislabeled$ examples are forgotten quickly, and seemingly $rare$ examples are forgotten comparatively slowly. SSFT can (i) help to identify mislabeled samples, the removal of which improves generalization; and (ii) provide insights about failure modes.
  arXiv  Detail & Related papers  (2022-10-26T21:03:46Z)
• Bias Mimicking: A Simple Sampling Approach for Bias Mitigation [57.17709477668213]
  We introduce a new class-conditioned sampling method: Bias Mimicking. Bias Mimicking improves underrepresented groups' accuracy of sampling methods by 3% over four benchmarks.
  arXiv  Detail & Related papers  (2022-09-30T17:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.