Fisher Widths: Local Learning Geometry and Anisotropic Recovery

• URL: http://arxiv.org/abs/2607.20578v1
• Date: Wed, 22 Jul 2026 10:07:03 GMT
• Title: Fisher Widths: Local Learning Geometry and Anisotropic Recovery
• Abstract summary: We study a pair of functionals: the primal Fisher width $w_G(T) = w(G1/2T)$, induced by the Fisher metric, and the inverse-Fisher width $w_G-1(T) = w(G-1/2T)$, induced by the inverse Fisher metric.<n>For Fisher-regular losses, we prove that the scale (w_G(H_r)/sqrt n) is attained on sufficiently small Fisher balls.<n>For sparse recovery, the resulting geometry depends not only on spars
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study Gaussian-width complexity on statistical manifolds through a pair of functionals: the primal Fisher width $w_G(T) = w(G^{1/2}T)$, induced by the Fisher metric, and the inverse-Fisher width $w_{G^{-1}}(T) = w(G^{-1/2}T)$, induced by the inverse Fisher metric. The two widths play complementary statistical roles. On the learning side, the Fisher width measures the size of local parameter fluctuations in the geometry induced by the Fisher information. For Fisher-regular losses, we prove that the scale \(w_G(H_r)/\sqrt n\) is attained on sufficiently small Fisher balls. On the recovery side, the inverse-Fisher width captures the effect of anisotropic Gaussian measurements whose covariance is determined by the inverse Fisher information. For sparse recovery, the resulting geometry depends not only on sparsity but also on the position of the active coordinates in the Fisher spectrum. We obtain a two-sided estimate for the corresponding statistical dimension, together with support-sensitive recovery estimates and a natural ordering of supports with different curvature profiles. Finally, we establish a sharp relation between the primal and inverse-Fisher widths. On any common compact coordinate set $T$, they satisfy \[ w_G(T)w_{G^{-1}}(T)\geq w(T)^2. \] Thus, Fisher anisotropy may transfer complexity from one geometry to the other, but cannot reduce both widths relative to the Euclidean scale.

Related papers

• Loss-Parameterized Fisher Width Along Learning Trajectories [0.0]
  We first derive an exact trace--shape factorization and a deterministic stability bound for fixed compact probes.<n>Results support a branchwise, rather than universal, loss parametrization of Fisher width.
  arXiv  Detail & Related papers  (2026-08-21T18:52:04Z)
• DeepLoop: Depth Scaling for Looped Transformers [91.7922038545625]
  Looped Transformers scale sequential computation by applying a compact stack of physical blocks for multiple rounds.<n>We formalize this tied-depth effect through a first-order perturbation bound controlled by a visit-alignment coefficient.<n>The resulting method, textbfDeepLoop, keeps the Post-LN DeepNorm architecture and sets $= (2N)1/2$ and $=(8N)-1/2$ for unrolled depth.
  arXiv  Detail & Related papers  (2026-07-15T06:37:05Z)
• Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence [67.12978375116599]
  We study an alternative objective: the reverse Fisher divergence, where the expectation is taken with respect to the student distribution.<n>We prove global convergence guarantees under a $widetilde(1)$-separation assumption on the target means.<n>Our rely on a new Lyapunov-based analysis of the gradient descent dynamics, showing that the reverse Fisher divergence has a much more favorable optimization landscape than the forward Fisher divergence.
  arXiv  Detail & Related papers  (2026-06-18T07:34:33Z)
• Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds [0.0]
  We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifold.<n>At a parameter point $$, Fisher width replaces the Euclidean identity by the local metric tensor $G()1/2$, measuring the Gaussian width of the Fisher-rescaled set.<n>We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric stability, and spectral comparison bounds with the Euclidean baseline.<n>As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which
  arXiv  Detail & Related papers  (2026-06-16T07:04:47Z)
• Fisher geometry reshapes the effect of incompatibility in multiparameter quantum estimation [1.0542145271875853]
  We show how the distribution of incompatibility across parameter planes affects its overall cost.<n>We also prove that at fixed sloppiness, or equivalently fixed Fisher volume, concentrating incompatibility into a single parameter plane reduces the optimized trade-off cost.<n>A qutrit $SU(2)$ encoding numerically confirms that states with larger incompatibility strength can nevertheless incur a smaller cost if the matching factor $G$ is sufficiently small.
  arXiv  Detail & Related papers  (2026-06-09T18:22:29Z)
• FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers [7.108048394322025]
  Activation steering methods modify intermediate representations of language models to control output behavior, but universally assume the activation space is Euclidean.<n>We show this assumption fails drastically: the local geometry induced by the model's own output behavior deviates from the Euclidean metric by over 97% in relative spectral norm on GPT-2.
  arXiv  Detail & Related papers  (2026-05-17T03:00:16Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Approximation and bounding techniques for the Fisher-Rao distances between parametric statistical models [7.070726553564701]
  We consider several numerically robust approximation and bounding techniques for the Fisher-Rao distances.<n>In particular, we obtain a generic method to guarantee an arbitrarily small additive error on the approximation.<n>We propose two new distances based either on the Fisher-Rao lengths of curves serving as proxies of Fisher-Rao geodesics.
  arXiv  Detail & Related papers  (2024-03-15T08:05:16Z)
• The Fisher-Rao geometry of CES distributions [50.50897590847961]
  The Fisher-Rao information geometry allows for leveraging tools from differential geometry. We will present some practical uses of these geometric tools in the framework of elliptical distributions.
  arXiv  Detail & Related papers  (2023-10-02T09:23:32Z)
• Bayesian Interpolation with Deep Linear Networks [92.1721532941863]
  Characterizing how neural network depth, width, and dataset size jointly impact model quality is a central problem in deep learning theory. We show that linear networks make provably optimal predictions at infinite depth. We also show that with data-agnostic priors, Bayesian model evidence in wide linear networks is maximized at infinite depth.
  arXiv  Detail & Related papers  (2022-12-29T20:57:46Z)
• Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence [100.6913091147422]
  Existing rotated object detectors are mostly inherited from the horizontal detection paradigm. In this paper, we are motivated to change the design of rotation regression loss from induction paradigm to deduction methodology.
  arXiv  Detail & Related papers  (2021-06-03T14:29:19Z)
• Debiased Sinkhorn barycenters [110.79706180350507]
  Entropy regularization in optimal transport (OT) has been the driver of many recent interests for Wasserstein metrics and barycenters in machine learning. We show how this bias is tightly linked to the reference measure that defines the entropy regularizer. We propose debiased Wasserstein barycenters that preserve the best of both worlds: fast Sinkhorn-like iterations without entropy smoothing.
  arXiv  Detail & Related papers  (2020-06-03T23:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.