Related papers: Phase Transition Frequency as a Training Time Predictor of Test Accuracy in ResNets

Phase Transition Frequency as a Training Time Predictor of Test Accuracy in ResNets

URL: http://arxiv.org/abs/2609.05194v1
Date: Fri, 04 Sep 2026 14:28:57 GMT
Title: Phase Transition Frequency as a Training Time Predictor of Test Accuracy in ResNets
Abstract summary: The number of discrete class-separability jumps observed during ResNet finetuning is examined empirically.<n>Across 75 experiments, a strong within-dataset negative correlation is obtained on standard i.i.d. classification benchmarks.
Score: 0.0
License: http://creativecommons.org/licenses/by/4.0/
Abstract: The number of discrete class-separability jumps observed during ResNet finetuning is examined empirically as a predictor of final test accuracy. Across 75 experiments spanning four benchmarks (CIFAR-10, CIFAR-100, TinyImageNet, and CIFAR-10-C) and three architectures (ResNet-18, ResNet-50, and ResNet-101), with five to ten seeds per configuration, a strong within-dataset negative correlation is obtained on standard i.i.d. classification benchmarks: $r = -0.84$ on CIFAR-10 ($p < 10^{-8}$, $n = 30$) and $r = -0.87$ on CIFAR-100 ($p < 10^{-5}$, $n = 15$). Under distributional stress, the relationship attenuates: TinyImageNet yields $r = -0.45$, and the CIFAR-10-C corruption benchmark yields $r = -0.19$. Two additional analyses discipline the empirical claim. A partial correlation controlling for architecture depth, treated as a linear covariate, shows that on CIFAR-100 the transition count retains statistically significant predictive power ($r_{\mathrm{partial}} = -0.69$, $p = 0.007$); the corresponding result under the stricter categorical conditioning is not established at $n = 15$. A comparison against six alternative training-curve signals shows that transition count achieved the strongest correlation among the evaluated signals on CIFAR-100 and one of the strongest on CIFAR-10, but is dominated by other signals on the two stressed benchmarks. The comparison is restricted to training-curve-level signals; comparisons against effective rank, Hessian sharpness, Fisher information, margin, and neural-collapse measures, which are the strongest competitors in the current literature, are not part of the present study and remain open. The observation is presented as an in-distribution training-quality probe among a family of candidate probes, and an inexpensive detection procedure suitable for logging alongside a standard training loop is provided.

Related papers

How Much Capacity Does EEG Denoising Need? Ultra-Compact Networks reveal Benchmark Saturation and Metric-Utility Gap [0.0]
Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable.<n>We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwiseseparable convolutional U-Net.
arXiv Detail & Related papers (2026-06-07T12:17:25Z)
Knot-10:A Tightness-Stratified Benchmark for Real-World Knot Classification with Topological Difficulty Analysis [0.0]
We introduce the Knots-10 benchmark, comprising 1,440 images with a deployment-oriented split that trains on loosely tied knots and tests on tightly dressed ones.<n>Swin-T and TransFG both average 97.2% accuracy; PMG scores 94.5%, consistent with the hypothesis that jigsaw shuffling disrupts crossing continuity.<n>We propose TACA regularization, which improves embedding-topology alignment without improving classification accuracy.
arXiv Detail & Related papers (2026-03-24T14:50:34Z)
Open Horizons: Evaluating Deep Models in the Wild [0.0]
We present a unified experimental study spanning open-set recognition (OSR) and few-shot class-incremental learning (FSCIL) on CIFAR-10.<n>For OSR, we compare three pretrained frozen visual encoders: ResNet-50, ConvNeXt-Tiny and CLIP ViT-B/16,using a linear probe and four post-hoc scoring functions, namely MSP, Energy, Mahalanobis and kNN.<n>For FSCIL, we compare modified SPPR, OrCo, and ConCM using partially frozen ResNet-50 across 1-, 5-, and 10-shot scenarios
arXiv Detail & Related papers (2025-12-13T03:03:59Z)
Exploring the Hierarchical Reasoning Model for Small Natural-Image Classification Without Augmentation [51.56484100374058]
It is evaluated on MNIST, CIFAR-10, and CIFAR-100 under a deliberately raw regime.<n>It is concluded that, for small-resolution image classification without augmentation, HRM is not competitive with even simple convolutional architectures.
arXiv Detail & Related papers (2025-10-04T01:22:41Z)
Variational Supervised Contrastive Learning [50.79938854370321]
We propose Variational Supervised Contrastive Learning (VarCon), which reformulates supervised contrastive learning as variational inference over latent class variables.<n>VarCon achieves state-of-the-art performance for contrastive learning frameworks, reaching 79.36% Top-1 accuracy on ImageNet-1K and 78.29% on CIFAR-100 with a ResNet-50 encoder.
arXiv Detail & Related papers (2025-06-09T04:19:12Z)
Randomness Helps Rigor: A Probabilistic Learning Rate Scheduler Bridging Theory and Deep Learning Practice [7.494722456816369]
We propose a probabilistic learning rate scheduler (PLRS)<n>PLRS does not conform to the monotonically decreasing condition, with provable convergence guarantees.<n>We show that PLRS performs as well as or better than existing state-of-the-art learning rate schedulers in terms of convergence as well as accuracy.
arXiv Detail & Related papers (2024-07-10T12:52:24Z)
Improved techniques for deterministic l2 robustness [63.34032156196848]
Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
arXiv Detail & Related papers (2022-11-15T19:10:12Z)
MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning [12.365801596593936]
We model our pre-training task as a binary classification problem to induce an implicit contrastive effect.<n>Unlike existing methods, the proposed loss function optimize the mutual information in positive and negative pairs.<n>The proposed method outperforms SOTA self-supervised contrastive frameworks on benchmark datasets.
arXiv Detail & Related papers (2021-11-24T17:51:29Z)
Adversarial Training with Rectified Rejection [114.83821848791206]
We propose to use true confidence (T-Con) as a certainty oracle, and learn to predict T-Con by rectifying confidence. We prove that under mild conditions, a rectified confidence (R-Con) rejector and a confidence rejector can be coupled to distinguish any wrongly classified input from correctly classified ones.
arXiv Detail & Related papers (2021-05-31T08:24:53Z)
Semi-supervised Contrastive Learning with Similarity Co-calibration [72.38187308270135]
We propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL) SsCL combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning. We show that SsCL produces more discriminative representation and is beneficial to few shot learning.
arXiv Detail & Related papers (2021-05-16T09:13:56Z)
Second-Order Provable Defenses against Adversarial Attacks [63.34032156196848]
We show that if the eigenvalues of the network are bounded, we can compute a certificate in the $l$ norm efficiently using convex optimization. We achieve certified accuracy of 5.78%, and 44.96%, and 43.19% on 2,59% and 4BP-based methods respectively.
arXiv Detail & Related papers (2020-06-01T05:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.