Shorting Dynamics and Structured Kernel Regularization

• URL: http://arxiv.org/abs/2512.04874v1
• Date: Thu, 04 Dec 2025 15:02:42 GMT
• Title: Shorting Dynamics and Structured Kernel Regularization
• Authors: James Tian,
• Abstract summary: The induced sequence of positive operators is monotone, admits an exact residual decomposition, and converges to the classical shorted operator.<n>This gives a unified operator-analytic approach to invariant kernel construction and structured regularization in data analysis.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper develops a nonlinear operator dynamic that progressively removes the influence of a prescribed feature subspace while retaining maximal structure elsewhere. The induced sequence of positive operators is monotone, admits an exact residual decomposition, and converges to the classical shorted operator. Transporting this dynamic to reproducing kernel Hilbert spaces yields a corresponding family of kernels that converges to the largest kernel dominated by the original one and annihilating the given subspace. In the finite-sample setting, the associated Gram operators inherit a structured residual decomposition that leads to a canonical form of kernel ridge regression and a principled way to enforce nuisance invariance. This gives a unified operator-analytic approach to invariant kernel construction and structured regularization in data analysis.

Related papers

• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• A General Weighting Theory for Ensemble Learning: Beyond Variance Reduction via Spectral and Geometric Structure [0.0]
  This paper develops a general weighting theory for ensemble learning.<n>We formalize ensembles as linear operators acting on a hypothesis space.<n>We show how non-uniform, structured weights can outperform uniform averaging.
  arXiv  Detail & Related papers  (2025-12-25T08:51:01Z)
• Learning Operators by Regularized Stochastic Gradient Descent with Operator-valued Kernels [5.663076715852465]
  We investigate regularized convergence encoder descent (SGD) algorithms for estimating nonlinear operators from a Polish space to a separable Hilbert space.<n>We present a new technique for deriving bounds with high probability for general SGD schemes.
  arXiv  Detail & Related papers  (2025-04-25T08:57:38Z)
• High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization [83.06112052443233]
  This paper studies kernel ridge regression in high dimensions under covariate shifts. By a bias-variance decomposition, we theoretically demonstrate that the re-weighting strategy allows for decreasing the variance. For bias, we analyze the regularization of the arbitrary or well-chosen scale, showing that the bias can behave very differently under different regularization scales.
  arXiv  Detail & Related papers  (2024-06-05T12:03:27Z)
• Entrywise error bounds for low-rank approximations of kernel matrices [55.524284152242096]
  We derive entrywise error bounds for low-rank approximations of kernel matrices obtained using the truncated eigen-decomposition. A key technical innovation is a delocalisation result for the eigenvectors of the kernel matrix corresponding to small eigenvalues. We validate our theory with an empirical study of a collection of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-05-23T12:26:25Z)
• Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum [6.749750044497731]
  We prove the phenomena of tempered overfitting and catastrophic overfitting under the sub-Gaussian design assumption. We also identify that the independence of the features plays an important role in guaranteeing tempered overfitting.
  arXiv  Detail & Related papers  (2024-02-02T10:36:53Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks [3.566568169425391]
  We show that with increased depth, node representations become dominated by a low-dimensional subspace that depends on the aggregation function but not on the feature transformations. For all aggregation functions, the rank of the node representations collapses, resulting in over-smoothing for particular aggregation functions.
  arXiv  Detail & Related papers  (2023-08-31T15:22:31Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Towards Understanding Generalization via Decomposing Excess Risk Dynamics [13.4379473119565]
  We analyze the generalization dynamics to derive algorithm-dependent bounds, e.g., stability. Inspired by the observation that neural networks show a slow convergence rate when fitting noise, we propose decomposing the excess risk dynamics. Under the decomposition framework, the new bound accords better with the theoretical and empirical evidence compared to the stability-based bound and uniform convergence bound.
  arXiv  Detail & Related papers  (2021-06-11T03:42:45Z)
• The kernel perspective on dynamic mode decomposition [4.051099980410583]
  This manuscript revisits theoretical assumptions concerning dynamic mode decomposition (DMD) of Koopman operators. Counterexamples that illustrate restrictiveness of the assumptions are provided for each of the assumptions. New framework for DMD requires only densely defined Koopman operators over RKHSs.
  arXiv  Detail & Related papers  (2021-05-31T21:20:01Z)
• Models of zero-range interaction for the bosonic trimer at unitarity [91.3755431537592]
  We present the construction of quantum Hamiltonians for a three-body system consisting of identical bosons mutually coupled by a two-body interaction of zero range. For a large part of the presentation, infinite scattering length will be considered.
  arXiv  Detail & Related papers  (2020-06-03T17:54:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.