Spectral Rank Certification for Foundation Model Adapters

• URL: http://arxiv.org/abs/2608.15351v1
• Date: Sat, 15 Aug 2026 18:11:20 GMT
• Title: Spectral Rank Certification for Foundation Model Adapters
• Abstract summary: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity.<n>This article develops a finite-sample framework for inferring effective rank structure in public foundation-model adapters.<n>In an audit of 26 public adapters, 684 modules, six architecture families, and 31,770 public-checkpoint spectra rows, calibrated effective rank is typically much smaller than nominal rank.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity. This article develops a finite-sample framework for inferring effective rank structure in public foundation-model adapters. The theoretical core is an exact chi-square divergence for the fixed-dimensional Gaussian rank-one reference experiment, with an unknown signal direction integrated under a rotation-invariant reference prior. The resulting series yields a computable finite-sample Le Cam bound at concrete layer sizes, an explicit remainder bound for numerical truncation, and the rectangular Baik-Ben Arous-Peche (BBP) limit. A compact-manifold Laplace expansion shows that finite-sample likelihood evidence also depends on leading spectral gaps through the factor $s_1^{|m-n|}\prod_{i\ge2}(s_1^2-s_i^2)$, motivating joint calibration of clustered singular values. Building on these results, we introduce an empirical-null workflow for PEFT LoRA adapters: factor reconstruction, Monte Carlo $p$-values, stagewise and block testing, and module-wise and corpus-level BH reporting. In an audit of 26 public adapters, 684 modules, six architecture families, and 31,770 public-checkpoint spectra rows, calibrated effective rank is typically much smaller than nominal rank and differs systematically from 95\% energy retention. A measured RoBERTa-RTE slice on $n=24$ examples illustrates the measurement path from calibrated ranks to task evaluation, without treating the slice as a utility study. The main empirical finding is that calibrated effective rank is usually far below nominal rank, and that energy retention and statistical surprise answer different questions.

Related papers

• Scaling-Score Conformal Prediction for Multi-Target Regression [0.42970700836450476]
  Multi-target regression requires a model to simultaneously predict several related outputs.<n>We propose the scaling-score conformal method, which is model-agnostic (requires only component-wise absolute residuals)
  arXiv  Detail & Related papers  (2026-09-15T12:23:57Z)
• Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning [52.452902154360565]
  SHOPCA (Shape Operator-based Principal Component Analysis) is a novel method for unsupervised metric learning and dimensionality reduction.<n>We evaluate SHOPCA on more than 50 real-world benchmark datasets.
  arXiv  Detail & Related papers  (2026-08-15T16:37:47Z)
• Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference [0.5076419064097734]
  Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA)<n>PLGA replaces the fixed bilinear form of scaled dotproduct attention (SDPA) with a learned, input-generated bilinear operator $G_LM$.<n>PLGA contains SDPA exactly at $G_LM=I$; $A_LM$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_LM$.
  arXiv  Detail & Related papers  (2026-08-10T22:45:22Z)
• DA-CASE: reusable measurements for adaptive quantum subspaces [0.0]
  We present theDyadic Adaptive Clifford-Algebra Subspace Eigensolver (DA-CASE)<n>We make that trade explicit on a frozen eight-qubit H$_4$ Hamiltonian.<n>Independently, a dyadic commuting hierarchy reduces thedeterminant bank from 913 qubit-wise-commuting settings to 64 fully commutingsettings.
  arXiv  Detail & Related papers  (2026-08-09T14:35:52Z)
• Finite Constant Frontiers and Auditable Regret Certificates for Average-Reward Reinforcement Learning [0.9558392439655014]
  We introduce a constant-aware comparison protocol and derive an explicit finite lower certificate for communicating MDPs.<n>A common closed-form envelope improves the published coefficient $0.015$ across a finite frontier.
  arXiv  Detail & Related papers  (2026-08-07T19:28:58Z)
• Statistical Inference for Rank Allocation in Low-Rank Adaptation [54.886931928255564]
  Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models.<n>We propose StatLoRA, a statistical inference-based rank allocation method.<n>We show that StatLoRA achieves comparable or better performance than vanilla LoRA, AdaLoRA, and I-LoRA under matched rank budgets.
  arXiv  Detail & Related papers  (2026-07-22T14:25:59Z)
• SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning [57.85676271833619]
  Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning.<n>As the rank increases, more principal singular directions are preserved, which generally improves the model's performance.<n>We propose SMoA, a textbfSpectrum textbfModulation textbfAdapter that enlarges the accessible family of spectrum-aware updates.
  arXiv  Detail & Related papers  (2026-05-20T13:19:28Z)
• Milestones over Outcome: Unlocking Geometric Reasoning with Sub-Goal Verifiable Reward [67.00373428443879]
  We introduce a paradigm shift towards subgoal-level evaluation and learning.<n>We first construct GeoGoal, a benchmark synthesized via a rigorous formal verification data engine.<n>We propose the Sub-Goal Verifiable Reward (SGVR) framework, which replaces sparse signals with dense rewards based on the Skeleton Rate.
  arXiv  Detail & Related papers  (2026-01-08T16:17:56Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• Spectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression [1.342834401139078]
  We introduce SpectrumAware Debiasing, a novel method for high-dimensional regression. Our approach applies to problems with structured, heavy tails, and low-rank structures. We demonstrate our method through simulated and real data experiments.
  arXiv  Detail & Related papers  (2023-09-14T15:58:30Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.