Loss-Parameterized Fisher Width Along Learning Trajectories

• URL: http://arxiv.org/abs/2608.21561v1
• Date: Fri, 21 Aug 2026 18:52:04 GMT
• Title: Loss-Parameterized Fisher Width Along Learning Trajectories
• Abstract summary: We first derive an exact trace--shape factorization and a deterministic stability bound for fixed compact probes.<n>Results support a branchwise, rather than universal, loss parametrization of Fisher width.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fisher width measures the Gaussian width of a probe set after deformation by the local Fisher geometry. We study its evolution along learning trajectories and ask when training loss can serve as an effective coordinate for this quantity. We first derive an exact trace--shape factorization and a deterministic stability bound for fixed compact probes. In a population Gaussian-teacher logistic model, the teacher-aligned state is extremal on every loss level below $\log 2$: it has minimal parameter norm and maximizes both Fisher trace and Euclidean-ball Fisher width. We then show that population gradient flow asymptotically selects this branch, with explicit rates for the aligned and orthogonal coordinates. This yields, for $d\geq2$, \[ \frac{w_F(B_2^d;θ(t))} {\sqrt{L(θ(t))}} \longrightarrow \frac{\sqrt6}π\mathbb E[χ_{d-1}]. \] Controlled full-Fisher experiments support the matched-loss branch and the population predictions. In a nonlinear MLP with a diagonal model-Fisher approximation, GD and SGD remain close at matched loss, whereas Adam follows a substantially displaced branch; the fixed probes tested retain highly similar temporal shapes. These results support a branchwise, rather than universal, loss parametrization of Fisher width.

Related papers

• Fisher Widths: Local Learning Geometry and Anisotropic Recovery [0.0]
  We study a pair of functionals: the primal Fisher width $w_G(T) = w(G1/2T)$, induced by the Fisher metric, and the inverse-Fisher width $w_G-1(T) = w(G-1/2T)$, induced by the inverse Fisher metric.<n>For Fisher-regular losses, we prove that the scale (w_G(H_r)/sqrt n) is attained on sufficiently small Fisher balls.<n>For sparse recovery, the resulting geometry depends not only on spars
  arXiv  Detail & Related papers  (2026-07-22T10:07:03Z)
• Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence [67.12978375116599]
  We study an alternative objective: the reverse Fisher divergence, where the expectation is taken with respect to the student distribution.<n>We prove global convergence guarantees under a $widetilde(1)$-separation assumption on the target means.<n>Our rely on a new Lyapunov-based analysis of the gradient descent dynamics, showing that the reverse Fisher divergence has a much more favorable optimization landscape than the forward Fisher divergence.
  arXiv  Detail & Related papers  (2026-06-18T07:34:33Z)
• Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds [0.0]
  We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifold.<n>At a parameter point $$, Fisher width replaces the Euclidean identity by the local metric tensor $G()1/2$, measuring the Gaussian width of the Fisher-rescaled set.<n>We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric stability, and spectral comparison bounds with the Euclidean baseline.<n>As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which
  arXiv  Detail & Related papers  (2026-06-16T07:04:47Z)
• Geometric and Spectral Alignment for Deep Neural Network I [12.227949990332363]
  We prove deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.<n>Full-rank factors are mapped from $mathrmGL(d)$ to the positive cone by $Amapsto Atop A$, then to ordered eigenvalue data.<n>We prove approximate-power-law and metric-chart versions, converse lower bounds, Fisher--KL/Bures action estimates, and near-identity expansions for normalized residual chains.
  arXiv  Detail & Related papers  (2026-05-04T00:07:24Z)
• Fisher-Geometric Diffusion in Stochastic Gradient Descent: Optimal Rates, Oracle Complexity, and Information-Theoretic Limits [0.0]
  We develop a theory of gradient descent in which mini-batch noise is an intrinsic, loss-induced matrix.<n>We prove oracle-complexity guarantees for epsilon-stationarity in the Fisher dual norm.
  arXiv  Detail & Related papers  (2026-03-02T21:57:09Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Unbiased Kinetic Langevin Monte Carlo with Inexact Gradients [0.8749675983608172]
  We present an unbiased method for posterior means based on kinetic Langevin dynamics. Our proposed estimator is unbiased, attains finite variance, and satisfies a central limit theorem. Our results demonstrate that in large-scale applications, the unbiased algorithm we present can be 2-3 orders of magnitude more efficient than the gold-standard" randomized Hamiltonian Monte Carlo.
  arXiv  Detail & Related papers  (2023-11-08T21:19:52Z)
• The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks [53.95175206863992]
  We study the type of solutions to which gradient descent converges when used to train a single hidden-layer multivariate ReLU network with the quadratic loss. We prove that although shallow ReLU networks are universal approximators, stable shallow networks are not.
  arXiv  Detail & Related papers  (2023-06-30T09:17:39Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Optimal Robust Linear Regression in Nearly Linear Time [97.11565882347772]
  We study the problem of high-dimensional robust linear regression where a learner is given access to $n$ samples from the generative model $Y = langle X,w* rangle + epsilon$ We propose estimators for this problem under two settings: (i) $X$ is L4-L2 hypercontractive, $mathbbE [XXtop]$ has bounded condition number and $epsilon$ has bounded variance and (ii) $X$ is sub-Gaussian with identity second moment and $epsilon$ is
  arXiv  Detail & Related papers  (2020-07-16T06:44:44Z)
• A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent [85.77233010209368]
  This article characterizes the exacts of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples $n$ is all large and comparable. This analysis also provides accurate estimates of training and test regression errors for large $n,p,N$.
  arXiv  Detail & Related papers  (2020-06-09T02:05:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.