TPV: Parameter Perturbations Through the Lens of Test Prediction Variance

• URL: http://arxiv.org/abs/2512.11089v1
• Date: Thu, 11 Dec 2025 20:04:33 GMT
• Title: TPV: Parameter Perturbations Through the Lens of Test Prediction Variance
• Abstract summary: Test prediction variance (TPV) is the first-order sensitivity of model outputs to perturbations around a trained solution.<n>TPV is a fully label-free object whose trace form separates the geometry of the trained model from the specific perturbation mechanism.
• Score: 1.9443009938188445
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We identify test prediction variance (TPV) -- the first-order sensitivity of model outputs to parameter perturbations around a trained solution -- as a unifying quantity that links several classical observations about generalization in deep networks. TPV is a fully label-free object whose trace form separates the geometry of the trained model from the specific perturbation mechanism, allowing a broad family of parameter perturbations like SGD noise, label noise, finite-precision noise, and other post-training perturbations to be analyzed under a single framework. Theoretically, we show that TPV estimated on the training set converges to its test-set value in the overparameterized limit, providing the first result that prediction variance under local parameter perturbations can be inferred from training inputs alone. Empirically, TPV exhibits a striking stability across datasets and architectures -- including extremely narrow networks -- and correlates well with clean test loss. Finally, we demonstrate that modeling pruning as a TPV perturbation yields a simple label-free importance measure that performs competitively with state-of-the-art pruning methods, illustrating the practical utility of TPV. Code available at github.com/devansharpit/TPV.

Related papers

• PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective [52.693471818837395]
  We argue that PEFT should be assessed through the stability-plasticity dilemma.<n>We introduce PEFT-Arena, a benchmark that measures downstream performance and general capability retention.<n>Inspired by this, we present case studies of a post-hoc improvement with path-wise rewinding.
  arXiv  Detail & Related papers  (2026-05-27T17:59:51Z)
• $α$-TCAV: A Unified Framework for Testing with Concept Activation Vectors [55.54306267760148]
  Concept Vectors (CAVs) are a tool for concept-based explainability in deep learning.<n>We analyze the nature of CAVs and the Activation Testing with CAVs (TCAV) method.<n>We introduce $$-TCAV, a generalized framework that replaces the indicator with a parameterized smooth function.
  arXiv  Detail & Related papers  (2026-05-15T07:22:59Z)
• FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection [63.91351553178842]
  FACTOR is a training-free Test-time adaptation for Open-vocabulary object detection.<n>It quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions.<n>Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods.
  arXiv  Detail & Related papers  (2026-05-05T02:31:18Z)
• Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation [21.687249397347017]
  We propose the Evidential Transformation Network (ETN), a lightweight post-hoc module that converts a pretrained predictor into an evidential model.<n>ETN consistently improves uncertainty estimation over post-hoc baselines while preserving accuracy and adding only minimal computational overhead.
  arXiv  Detail & Related papers  (2026-04-09T16:09:47Z)
• Visual prompting reimagined: The power of the Activation Prompts [72.85146015928626]
  We introduce a concept, termed activation prompt (AP), which extends the scope of the input-level VP.<n>AP is closely related to normalization tuning in convolutional neural networks and vision transformers.<n>We show that AP is closely related to normalization tuning in convolutional neural networks and vision transformers.
  arXiv  Detail & Related papers  (2026-04-07T20:28:24Z)
• Sharp Convergence Rates for Masked Diffusion Models [53.117058231393834]
  We develop a total-variation based analysis for the Euler method that overcomes limitations.<n>Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees.<n>Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS.
  arXiv  Detail & Related papers  (2026-02-26T00:47:51Z)
• Joint Embedding Variational Bayes [0.08594140167290097]
  Variational Joint Embedding (VJE) is a framework that synthesizes joint embedding and variational inference.<n>VJE enables self-supervised learning of probabilistic representations in a reconstruction-free, non-contrastive setting.
  arXiv  Detail & Related papers  (2026-02-05T13:18:53Z)
• Temporal Pair Consistency for Variance-Reduced Flow Matching [13.328987133593154]
  Temporal Pair Consistency (TPC) is a lightweight variance-reduction principle that couples velocity predictions at paired timesteps along the same probability path.<n>Instantiated within flow matching, TPC improves sample quality and efficiency across CIFAR-10 and ImageNet at multiple resolutions.
  arXiv  Detail & Related papers  (2026-02-04T00:05:21Z)
• Long-horizon Reasoning Agent for Olympiad-Level Mathematical Problem Solving [65.02106674311908]
  This paper introduces Intern-S1-MO, a long-horizon math agent that conducts multi-round hierarchical reasoning.<n>By maintaining a compact memory in the form of lemmas, Intern-S1-MO can more freely explore the lemma-rich reasoning spaces.<n> Experiments show that Intern-S1-MO can obtain 26 out of 35 points on the non-geometry problems of IMO2025, matching the performance of silver medalists.
  arXiv  Detail & Related papers  (2025-12-11T15:26:28Z)
• TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbation [11.334867025651233]
  We propose TULiP, a theoretically-driven uncertainty estimator for OOD detection.<n>Our approach considers a hypothetical perturbation applied to the network before convergence.<n>Our method exhibits state-of-the-art performance, particularly for near-distribution samples.
  arXiv  Detail & Related papers  (2025-05-22T17:16:41Z)
• TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation [3.7024647541541014]
  Post-training quantization (PTQ) reduces excessive hardware cost by quantizing full-precision models into lower bit representations on a tiny calibration set.<n>Traditional PTQ methods typically encounter failure in dynamic and ever-changing real-world scenarios.<n>We propose a novel and stable quantization process for test-time adaptation (TTA), dubbed TTAQ, to address the performance degradation of traditional PTQ.
  arXiv  Detail & Related papers  (2024-12-13T06:34:59Z)
• Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods [59.779795063072655]
  Chain-of-Thought (CoT) prompting and its variants have gained popularity as effective methods for solving multi-step reasoning problems. We analyze CoT prompting from a statistical estimation perspective, providing a comprehensive characterization of its sample complexity.
  arXiv  Detail & Related papers  (2024-08-25T04:07:18Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [65.21599711087538]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample.<n>Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications.<n>We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• CC-Cert: A Probabilistic Approach to Certify General Robustness of Neural Networks [58.29502185344086]
  In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks. It is important to provide provable guarantees for deep learning models against semantically meaningful input transformations. We propose a new universal probabilistic certification approach based on Chernoff-Cramer bounds.
  arXiv  Detail & Related papers  (2021-09-22T12:46:04Z)
• Controlling for sparsity in sparse factor analysis models: adaptive latent feature sharing for piecewise linear dimensionality reduction [2.896192909215469]
  We propose a simple and tractable parametric feature allocation model which can address key limitations of current latent feature decomposition techniques. We derive a novel adaptive Factor analysis (aFA), as well as, an adaptive probabilistic principle component analysis (aPPCA) capable of flexible structure discovery and dimensionality reduction. We show that aPPCA and aFA can infer interpretable high level features both when applied on raw MNIST and when applied for interpreting autoencoder features.
  arXiv  Detail & Related papers  (2020-06-22T16:09:11Z)
• A generative adversarial network approach to calibration of local stochastic volatility models [2.1485350418225244]
  We propose a fully data-driven approach to calibrate local volatility (LSV) models. We parametrize the leverage function by a family of feed-forward neural networks and learn their parameters directly from the available market option prices. This should be seen in the context of neural SDEs and (causal) generative adversarial networks.
  arXiv  Detail & Related papers  (2020-05-05T21:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.