Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres

• URL: http://arxiv.org/abs/2511.02625v2
• Date: Thu, 06 Nov 2025 02:21:26 GMT
• Title: Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres
• Authors: Xinliang Liu, Tong Mao, Jinchao Xu,
• Abstract summary: We present an estimation of the condition numbers of the emphmass and emphstiffness matrices arising from shallow ReLU$k$ neural networks.<n>This spectral analysis establishes a precise correspondence between the approximation power of the network and its stability.
• Score: 7.864201093845001
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present an estimation of the condition numbers of the \emph{mass} and \emph{stiffness} matrices arising from shallow ReLU$^k$ neural networks defined on the unit sphere~$\mathbb{S}^d$. In particular, when $\{\theta_j^*\}_{j=1}^n \subset \mathbb{S}^d$ is \emph{antipodally quasi-uniform}, the condition number is sharp. Indeed, in this case, we obtain sharp asymptotic estimates for the full spectrum of eigenvalues and characterize the structure of the corresponding eigenspaces, showing that the smallest eigenvalues are associated with an eigenbasis of low-degree polynomials while the largest eigenvalues are linked to high-degree polynomials. This spectral analysis establishes a precise correspondence between the approximation power of the network and its numerical stability.

Related papers

• Complex Eigenvalues in a pseudo-Hermitian \{eta}-Laguerre ensemble [0.0]
  We investigate an ensemble of unstable matrices iso to the beta-Laguerre ensemble.<n> Introducing a small non-Hermitian perturbation breaks the symmetry and drives the eigenvalues into the complex plane.<n>The behavior of these eigenvalues is analyzed in the large matrix-size limit, and our theoretical predictions are supported by numerical simulations.
  arXiv  Detail & Related papers  (2025-11-12T00:27:49Z)
• Lipschitz Bounds for Persistent Laplacian Eigenvalues under One-Simplex Insertions [0.0]
  We prove a uniform Lipschitz bound for Persistent Laplacians.<n>We deliver the first eigenvalue-level guarantee for spectral topological data analysis.
  arXiv  Detail & Related papers  (2025-06-26T15:03:54Z)
• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)
• Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction [55.57072563835959]
  We propose an eigenvalue correction strategy that can free filters from the constraints of repeated eigenvalue inputs.<n>Concretely, the proposed eigenvalue correction strategy enhances the uniform distribution of eigenvalues, and improves the fitting capacity and expressive power of filters.
  arXiv  Detail & Related papers  (2024-01-28T08:12:00Z)
• Topological complexity of spiked random polynomials and finite-rank spherical integrals [2.1756081703276]
  In particular, we establish variational formulas for the exponentials of the average number of total critical points and the determinants of local parameters of a finite-rank spiked Gaussian Wigner matrix. The analysis is based on recent advances on finite-rank spherical integrals by [Guionnet, Husson] to study the large deviations of multi-rank spiked Gaussian Wigner matrices. There is an exact threshold for the external parameters such that, once exceeded, the complexity function vanishes into new regions in which the critical points are close to the given vectors.
  arXiv  Detail & Related papers  (2023-12-19T16:52:01Z)
• Symmetry & Critical Points for Symmetric Tensor Decomposition Problems [6.123324869194196]
  We consider the non optimization problem associated with the decomposition of a real symmetric tensor into a sum of rank-one terms.<n>Use is made of rich symmetry structure to construct infinite families of critical points represented by Puiseux series in the problem dimension.
  arXiv  Detail & Related papers  (2023-06-13T16:25:30Z)
• Deep neural network approximation of analytic functions [91.3755431537592]
  entropy bound for the spaces of neural networks with piecewise linear activation functions. We derive an oracle inequality for the expected error of the considered penalized deep neural network estimators.
  arXiv  Detail & Related papers  (2021-04-05T18:02:04Z)
• A simpler spectral approach for clustering in directed networks [1.52292571922932]
  We show that using the eigenvalue/eigenvector decomposition of the adjacency matrix is simpler than all common methods. We provide numerical evidence for the superiority of the Gaussian Mixture clustering over the widely used k-means algorithm.
  arXiv  Detail & Related papers  (2021-02-05T14:16:45Z)
• Stochastic Approximation for Online Tensorial Independent Component Analysis [98.34292831923335]
  Independent component analysis (ICA) has been a popular dimension reduction tool in statistical machine learning and signal processing. In this paper, we present a by-product online tensorial algorithm that estimates for each independent component.
  arXiv  Detail & Related papers  (2020-12-28T18:52:37Z)
• Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry [9.695960412426672]
  We analytically characterize the Hessian at various families of spurious minima. In particular, we prove that for $dge k$ standard Gaussian inputs: (a) of the $dk$ eigenvalues of the Hessian, $dk - O(d)$ concentrate near zero, (b) $Omega(d)$ of the eigenvalues grow linearly with $k$.
  arXiv  Detail & Related papers  (2020-08-04T20:08:35Z)
• A Concentration of Measure and Random Matrix Approach to Large Dimensional Robust Statistics [45.24358490877106]
  This article studies the emphrobust covariance matrix estimation of a data collection $X = (x_1,ldots,x_n)$ with $x_i = sqrt tau_i z_i + m$. We exploit this semi-metric along with concentration of measure arguments to prove the existence and uniqueness of the robust estimator as well as evaluate its limiting spectral distribution.
  arXiv  Detail & Related papers  (2020-06-17T09:02:26Z)
• Profile Entropy: A Fundamental Measure for the Learnability and Compressibility of Discrete Distributions [63.60499266361255]
  We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically, profile entropy a) determines the speed of estimating the distribution relative to the best natural estimator; b) characterizes the rate of inferring all symmetric properties compared with the best estimator over any label-invariant distribution collection; c) serves as the limit of profile compression.
  arXiv  Detail & Related papers  (2020-02-26T17:49:04Z)
• Convex Geometry and Duality of Over-parameterized Neural Networks [70.15611146583068]
  We develop a convex analytic approach to analyze finite width two-layer ReLU networks. We show that an optimal solution to the regularized training problem can be characterized as extreme points of a convex set. In higher dimensions, we show that the training problem can be cast as a finite dimensional convex problem with infinitely many constraints.
  arXiv  Detail & Related papers  (2020-02-25T23:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.