Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

• URL: http://arxiv.org/abs/2607.21823v1
• Date: Thu, 23 Jul 2026 21:18:41 GMT
• Title: Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA
• Abstract summary: We show that up to isotropic scaling, the Gaussian RBF kernel space is reproducing Hilbertally isometric to Euclidean space in the large bandwidth limit.<n>We also consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) as bandwidth.<n>Experiments over a diverse collection of data sets demonstrate that $$ provides a simple and reliable predictor of dataset-specific convergence behavior.
• Score: 4.061135251278187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit. This strongly suggests that kernel-based constructions reliant on metric properties of the RKHS will yield results for Gaussian RBF kernels that similarly approach those of linear kernels for large bandwidths. The asymptotic behavior of Gaussian CKA can be understood in this light. We further consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) PCA as bandwidth $σ\rightarrow \infty$. For a given data representation, both the RKHS feature embeddings and the orthogonal PCA eigenframes of the two kernel types differ asymptotically by a geometric similarity transformation, up to a residual of size $O \left (\fracρσ \right )^2$, where $ρ$ is a measure of geometric eccentricity of the representation, equal to the ratio of maximum to median pairwise distance between data examples. Experiments over a diverse collection of data sets demonstrate that $ρ$ provides a simple and reliable predictor of dataset-specific convergence behavior in the top principal directions.

Related papers

• Kernel ridge regression under power-law data: spectrum and generalization [10.479735444470535]
  We investigate high-dimensional kernel ridge regression (KRR) on i.i.d. Gaussian data with anisotropic power-law covariance.<n>This setting differs fundamentally from the classical source & capacity conditions for KRR, where power-law assumptions are typically imposed on the kernel eigen-spectrum itself.<n>To our knowledge, this is the first rigorous treatment of non-linear KRR under power-law data.
  arXiv  Detail & Related papers  (2025-10-06T12:58:35Z)
• Highly Adaptive Ridge [84.38107748875144]
  We propose a regression method that achieves a $n-2/3$ dimension-free L2 convergence rate in the class of right-continuous functions with square-integrable sectional derivatives. Har is exactly kernel ridge regression with a specific data-adaptive kernel based on a saturated zero-order tensor-product spline basis expansion. We demonstrate empirical performance better than state-of-the-art algorithms for small datasets in particular.
  arXiv  Detail & Related papers  (2024-10-03T17:06:06Z)
• Gaussian kernel expansion with basis functions uniformly bounded in $\mathcal{L}_{\infty}$ [0.6138671548064355]
  Kernel expansions are a topic of considerable interest in machine learning. Recent work in the literature has derived some of these results by assuming uniformly bounded basis functions in $mathcalL_infty$. Our main result is the construction on $mathbbR2$ of a Gaussian kernel expansion with weights in $ell_p$ for any $p>1$.
  arXiv  Detail & Related papers  (2024-10-02T10:10:30Z)
• Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime [11.655527064124074]
  In this work, we extend the study of kernel regression to the quadratic regime, where $n asymp d2$.<n>Specifically, we establish an operator norm approximation bound for the difference between the original kernel random matrix and a quadratic kernel random matrix.<n>We characterize the precise training and test errors for KRR in the quadratic regime when $n/d2$ converges to a non-zero constant.
  arXiv  Detail & Related papers  (2024-08-02T07:29:49Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Simplex Random Features [53.97976744884616]
  We present Simplex Random Features (SimRFs), a new random feature (RF) mechanism for unbiased approximation of the softmax and Gaussian kernels. We prove that SimRFs provide the smallest possible mean square error (MSE) on unbiased estimates of these kernels. We show consistent gains provided by SimRFs in settings including pointwise kernel estimation, nonparametric classification and scalable Transformers.
  arXiv  Detail & Related papers  (2023-01-31T18:53:39Z)
• Sinusoidal Sensitivity Calculation for Line Segment Geometries [0.0]
  This paper presents a closed-form solution to the sinusoidal coil sensitivity model proposed by Kern et al. It allows for precise computations of varied, simulated bias fields for ground-truth debias datasets.
  arXiv  Detail & Related papers  (2022-08-05T09:30:55Z)
• An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings [21.727073594338297]
  This study is motivated by applications in machine learning and statistics. We establish the weak limit of the empirical distribution of these random matrices in a scaling regime. Our results can be characterized as the free additive convolution between a Marchenko-Pastur law and a semicircle law.
  arXiv  Detail & Related papers  (2022-05-12T18:50:21Z)
• A Note on Optimizing Distributions using Kernel Mean Embeddings [94.96262888797257]
  Kernel mean embeddings represent probability measures by their infinite-dimensional mean embeddings in a reproducing kernel Hilbert space. We show that when the kernel is characteristic, distributions with a kernel sum-of-squares density are dense. We provide algorithms to optimize such distributions in the finite-sample setting.
  arXiv  Detail & Related papers  (2021-06-18T08:33:45Z)
• A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent [85.77233010209368]
  This article characterizes the exacts of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples $n$ is all large and comparable. This analysis also provides accurate estimates of training and test regression errors for large $n,p,N$.
  arXiv  Detail & Related papers  (2020-06-09T02:05:40Z)
• Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nystr\"om method [76.73096213472897]
  We develop techniques which exploit spectral properties of the data matrix to obtain improved approximation guarantees. Our approach leads to significantly better bounds for datasets with known rates of singular value decay. We show that both our improved bounds and the multiple-descent curve can be observed on real datasets simply by varying the RBF parameter.
  arXiv  Detail & Related papers  (2020-02-21T00:43:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.