Spectral Signatures of Data Quality: Eigenvalue Tail Index as a Diagnostic for Label Noise in Neural Networks

• URL: http://arxiv.org/abs/2603.27885v1
• Date: Sun, 29 Mar 2026 21:53:24 GMT
• Title: Spectral Signatures of Data Quality: Eigenvalue Tail Index as a Diagnostic for Label Noise in Neural Networks
• Abstract summary: We investigate whether spectral properties of neural network weight matrices can predict test accuracy.<n>Under controlled label noise variation, the tail index alpha of the eigenvalue distribution at the network's bottleneck layer predicts test accuracy with leave-one-out R2 = 0.984 (21 noise levels, 3 seeds per level)<n>We identify the information-processing bottleneck layer as the locus of this signature and connect the observations to the BBP phase transition in random matrix models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate whether spectral properties of neural network weight matrices can predict test accuracy. Under controlled label noise variation, the tail index alpha of the eigenvalue distribution at the network's bottleneck layer predicts test accuracy with leave-one-out R^2 = 0.984 (21 noise levels, 3 seeds per level), far exceeding all baselines: the best conventional metric (Frobenius norm of the optimal layer) achieves LOO R^2 = 0.149. This relationship holds across three architectures (MLP, CNN, ResNet-18) and two datasets (MNIST, CIFAR-10). However, under hyperparameter variation at fixed data quality (180 configurations varying width, depth, learning rate, and weight decay), all spectral and conventional measures are weak predictors (R^2 < 0.25), with simple baselines (global L_2 norm, LOO R^2 = 0.219) slightly outperforming spectral measures (tail alpha, LOO R^2 = 0.167). We therefore frame the tail index as a data quality diagnostic: a powerful detector of label corruption and training set degradation, rather than a universal generalization predictor. A noise detector calibrated on synthetic noise successfully identifies real human annotation errors in CIFAR-10N (9% noise detected with 3% error). We identify the information-processing bottleneck layer as the locus of this signature and connect the observations to the BBP phase transition in spiked random matrix models. We also report a negative result: the level spacing ratio <r> is uninformative for weight matrices due to Wishart universality.

Related papers

• Can You Trust Frozen Hematology Foundation Models under Acquisition Shift? [0.0]
  We audit 15 frozen encoders across four public single-cell acquisition domains.<n>In-domain linear-probe macro-F1 is saturated (0.98-0.997), yet cross-dataset macro-F1 drops 34-72%.<n>DinoBloom-L, the in-domain best, falls to 10th of 15 on the most-shifted target (MLL23), behind RedDino and several general and pathology encoders.
  arXiv  Detail & Related papers  (2026-08-25T20:55:57Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection [6.893608092072852]
  Denoising score matching (DSM) provides a way to learn data distributions by training a neural network to recover the score function.<n>We introduce kurtosis-based noise scaling (K-DSM), a per-feature scheme that sets noise levels from the shape of each marginal distribution.<n>On standard anomaly detection benchmarks, K-DSM achieves state-of-the-art performance in the semi-supervised setting.
  arXiv  Detail & Related papers  (2026-05-07T21:19:12Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• Zero-Shot Detection of Elastic Transient Morphology Across Physical Systems [0.0]
  We show that a representation learned from interferometric strain transients can act as a morphology-sensitive operator for unseen sensors.<n>Using a neural encoder trained exclusively on non-Gaussian instrumental glitches, we perform strict zero-shot anomaly analysis on rolling-element bearings.
  arXiv  Detail & Related papers  (2026-01-16T16:35:07Z)
• Why Machine Learning Models Systematically Underestimate Extreme Values II: How to Fix It with LatentNN [0.2700171473617699]
  Attenuation bias affects astronomical data-driven models.<n>We show that neural networks suffer from the same attenuation bias.<n>We introduce LatentNN, a method that jointly optimize network parameters and latent input values.
  arXiv  Detail & Related papers  (2025-12-29T01:59:10Z)
• Reliable Active Learning from Unreliable Labels via Neural Collapse Geometry [5.1511135538176]
  Active Learning (AL) promises to reduce annotation cost by prioritizing informative samples, yet its reliability is undermined when labels are noisy or when the data distribution shifts.<n>We propose Active Learning via Neural Collapse Geometry (NCAL-R), a framework that leverages the emergent geometric regularities of deep networks to counteract unreliable supervision.
  arXiv  Detail & Related papers  (2025-10-10T17:50:31Z)
• Chirp Localization via Fine-Tuned Transformer Model: A Proof-of-Concept Study [0.23020018305241333]
  Chirp-like patterns in EEG spectrograms are key biomarkers for seizure dynamics.<n>This study bridges this gap by fine-tuning a Vision Transformer (ViT) model and Low-Rank Adaptation (LoRA)<n>We generated 100000 spectrograms with chirp parameters, creating the first large-scale benchmark for chirp localization.
  arXiv  Detail & Related papers  (2025-03-24T14:27:07Z)
• How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm? [73.80001705134147]
  We provide an exact characterization of the expected generalization error (gen-error) for semi-supervised learning (SSL) with pseudo-labeling via the Gibbs algorithm. The gen-error is expressed in terms of the symmetrized KL information between the output hypothesis, the pseudo-labeled dataset, and the labeled dataset.
  arXiv  Detail & Related papers  (2022-10-15T04:11:56Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Conditioning Trick for Training Stable GANs [70.15099665710336]
  We propose a conditioning trick, called difference departure from normality, applied on the generator network in response to instability issues during GAN training. We force the generator to get closer to the departure from normality function of real samples computed in the spectral domain of Schur decomposition.
  arXiv  Detail & Related papers  (2020-10-12T16:50:22Z)
• Deep learning for gravitational-wave data analysis: A resampling white-box approach [62.997667081978825]
  We apply Convolutional Neural Networks (CNNs) to detect gravitational wave (GW) signals of compact binary coalescences, using single-interferometer data from LIGO detectors. CNNs were quite precise to detect noise but not sensitive enough to recall GW signals, meaning that CNNs are better for noise reduction than generation of GW triggers.
  arXiv  Detail & Related papers  (2020-09-09T03:28:57Z)
• Weak Detection in the Spiked Wigner Model with General Rank [13.45821655503426]
  We study the statistical decision process of detecting the signal from a signal+noise' type matrix model with an additive Wigner noise. We propose a hypothesis test based on the linear spectral statistics of the data matrix, which does not depend on the distribution of the signal or the noise.
  arXiv  Detail & Related papers  (2020-01-16T06:40:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.