No Validation, No Problem: Predicting Model Performance from a Single Gradient

• URL: http://arxiv.org/abs/2601.16874v1
• Date: Fri, 23 Jan 2026 16:30:11 GMT
• Title: No Validation, No Problem: Predicting Model Performance from a Single Gradient
• Abstract summary: We propose a validation-free checkpointing signal from a single forward-backward pass.<n>Across ImageNet-1k CNNs and Transformers, this proxy is strongly negative with Top-1 and positive with loss.
• Score: 1.9336815376402718
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a validation-free checkpointing signal from a single forward-backward pass: the Frobenius norm of the classifier-head gradient on one detached-feature batch, ||g||_F = ||dL/dW||_F. Across ImageNet-1k CNNs and Transformers, this proxy is strongly negative with Top-1 and positive with loss. Selecting the checkpoint with the minimum head gradient in a short tail window closes most of the gap to the oracle (4.24% +/- 2.00% with a universal setup, about 1.12% with light per-family tuning). For practical deployment, a head-scale normalization is more stable within classic CNN families (e.g., ResNets), while a feature-scale normalization works well for Transformers and modern CNNs. The same one-batch probe also predicts COCO detection/segmentation mAP. In diffusion (UNet/DDPM on CIFAR-10), it tracks progress and enables near-oracle tail-window selection; it is positively correlated with same-distribution probe MSE and negatively with FID (lower is better), so it can be used as a lightweight, label-free monitor. Validation labels are never used beyond reporting. The probe adds much less than 0.1% of an epoch and works as a drop-in for validation-free checkpoint selection and early stopping.

Related papers

• Interface-Variant Dynamics in Software Ecosystems: Resolver-Induced Selection and Adoption in Package Graphs [0.2864713389096699]
  This paper asks when that observation is a load-bearing software-engineering estimator rather than evolutionary relabeling.<n>We mine interface histories, audit npm, Central, PyPI, and crates.io package graphs, execute 2100 package-manager resolver probes, estimate an ecosystem-specific selection $s$ from clean conflict probabilities, and use that measured $s$ to forward evaluate a pairwise-comparison absorbing process on the observed package graph.
  arXiv  Detail & Related papers  (2026-06-30T15:29:39Z)
• Anticipating the Optimism Gap: Predicting Distribution-Shift Degradation of RF-Impairment Detectors from In-Distribution Statistics [0.0]
  Detectors for radio-frequency impairments are usually reported with a single AUC measured on the distribution they were tuned on.<n>We ask whether this optimism can be predicted before any out-of-distribution data is seen.
  arXiv  Detail & Related papers  (2026-06-20T14:10:59Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization [85.50560211492898]
  Test-time adaptation (TTA) may fail to improve or even harm the model performance when test data have mixed distribution shifts.<n>This is often a key obstacle preventing existing TTA methods from being deployed in the real world.<n>We propose a sharpness-aware and reliable entropy minimization method, called SAR, for stabilizing TTA from two aspects.
  arXiv  Detail & Related papers  (2025-09-05T10:03:00Z)
• Guess-and-Learn (G&L): Measuring the Cumulative Error Cost of Cold-Start Adaptation [0.11102988539107704]
  Evaluation of machine learning models typically emphasizes final accuracy, overlooking the cost of adaptation cumulative errors incurred while learning from scratch.<n>Guess-and- Learn (G&L) v1.0 addresses this gap by measuring cold-start adaptability - the total mistakes a model makes while sequentially labeling an unlabeled dataset.
  arXiv  Detail & Related papers  (2025-08-29T00:13:02Z)
• Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation [6.64353332639395]
  Test-time adaptation (TTA) aims to adapt a pre-trained model to a new test domain without access to source data after deployment. Existing approaches rely on self-training with pseudo-labels since ground-truth cannot be obtained from test data. We propose DPLOT, a simple yet effective TTA framework that consists of two components: (1) domain-specific block selection and (2) pseudo-label generation using paired-view images.
  arXiv  Detail & Related papers  (2024-04-17T00:21:36Z)
• Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization [64.62570402941387]
  We use a single test sample to adapt multi-modal prompts at test time by minimizing the feature distribution shift to bridge the gap in the test domain. Our method improves zero-shot top- 1 accuracy beyond existing prompt-learning techniques, with a 3.08% improvement over the baseline MaPLe.
  arXiv  Detail & Related papers  (2023-11-02T17:59:32Z)
• Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [58.789823426981044]
  We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions. Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
  arXiv  Detail & Related papers  (2023-03-25T08:56:21Z)
• DSLA: Dynamic smooth label assignment for efficient anchor-free object detection [18.043176234010517]
  Anchor-free detectors basically formulate object detection as dense classification and regression. It is common to introduce an individual prediction branch to estimate the quality of localization. The following inconsistencies are observed when we delve into the practices of classification and quality estimation.
  arXiv  Detail & Related papers  (2022-08-01T12:56:44Z)
• MutexMatch: Semi-supervised Learning with Mutex-based Consistency Regularization [36.019086181632005]
  We propose a mutex-based consistency regularization, namely Mutex, to utilize low-confidence samples in a novel way. MutexMatch achieves superior performance on multiple benchmark datasets, i.e., CIFAR-10, CIFAR-100, SVHN, STL-10, and mini-ImageNet.
  arXiv  Detail & Related papers  (2022-03-27T14:28:16Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• Deep learning for gravitational-wave data analysis: A resampling white-box approach [62.997667081978825]
  We apply Convolutional Neural Networks (CNNs) to detect gravitational wave (GW) signals of compact binary coalescences, using single-interferometer data from LIGO detectors. CNNs were quite precise to detect noise but not sensitive enough to recall GW signals, meaning that CNNs are better for noise reduction than generation of GW triggers.
  arXiv  Detail & Related papers  (2020-09-09T03:28:57Z)
• Tent: Fully Test-time Adaptation by Entropy Minimization [77.85911673550851]
  A model must adapt itself to generalize to new and different data during testing. In this setting of fully test-time adaptation the model has only the test data and its own parameters. We propose to adapt by test entropy minimization (tent): we optimize the model for confidence as measured by the entropy of its predictions.
  arXiv  Detail & Related papers  (2020-06-18T17:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.