Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

• URL: http://arxiv.org/abs/2603.00971v1
• Date: Sun, 01 Mar 2026 07:49:12 GMT
• Title: Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators
• Authors: Mike Nguyen, Nicole Mücke,
• Abstract summary: We extend prior results for Tikhonov regularization to a broad class of spectral regularization techniques.<n>We also generalize the setting to operator-valued kernels.
• Score: 0.7090165638014329
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.

Related papers

• Random feature approximation for general spectral methods [2.9388890036358104]
  This work extends previous results for Tikhonov regularization to a broad class of spectral regularization techniques.<n>We enable a theoretical analysis of neural networks and neural operators through the lens of the Neural Tangent Kernel (NTK) approach.
  arXiv  Detail & Related papers  (2025-06-19T13:00:17Z)
• Optimal Convergence Rates for Neural Operators [2.9388890036358104]
  We provide bounds on the number of hidden neurons and the number of second-stage samples necessary for generalization.<n>A key application of neural operators is learning surrogate maps for the solution operators of partial differential equations.
  arXiv  Detail & Related papers  (2024-12-23T12:31:38Z)
• Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning [11.165984223900104]
  This paper introduces the Kernel Neural Operator (KNO), a provably convergent operator-learning architecture.<n>The KNO decouples the choice of kernel from the numerical integration scheme (quadrature)<n> Numerical results demonstrate that on existing benchmarks the training and test accuracy of KNOs is comparable to or higher than popular operator learning techniques.
  arXiv  Detail & Related papers  (2024-06-30T19:28:12Z)
• An Exact Kernel Equivalence for Finite Classification Models [1.4777718769290527]
  We compare our exact representation to the well-known Neural Tangent Kernel (NTK) and discuss approximation error relative to the NTK. We use this exact kernel to show that our theoretical contribution can provide useful insights into the predictions made by neural networks.
  arXiv  Detail & Related papers  (2023-08-01T20:22:53Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Gradient Descent in Neural Networks as Sequential Learning in RKBS [63.011641517977644]
  We construct an exact power-series representation of the neural network in a finite neighborhood of the initial weights. We prove that, regardless of width, the training sequence produced by gradient descent can be exactly replicated by regularized sequential learning.
  arXiv  Detail & Related papers  (2023-02-01T03:18:07Z)
• Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK [86.45209429863858]
  We study training one-hidden-layer ReLU networks in the neural tangent kernel (NTK) regime. We show that the neural networks possess a different limiting kernel which we call textitbias-generalized NTK We also study various properties of the neural networks with this new kernel.
  arXiv  Detail & Related papers  (2023-01-01T02:11:39Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Uniform Generalization Bounds for Overparameterized Neural Networks [5.945320097465419]
  We prove uniform generalization bounds for overparameterized neural networks in kernel regimes. Our bounds capture the exact error rates depending on the differentiability of the activation functions. We show the equivalence between the RKHS corresponding to the NT kernel and its counterpart corresponding to the Mat'ern family of kernels.
  arXiv  Detail & Related papers  (2021-09-13T16:20:13Z)
• Random Features for the Neural Tangent Kernel [57.132634274795066]
  We propose an efficient feature map construction of the Neural Tangent Kernel (NTK) of fully-connected ReLU network. We show that dimension of the resulting features is much smaller than other baseline feature map constructions to achieve comparable error bounds both in theory and practice.
  arXiv  Detail & Related papers  (2021-04-03T09:08:12Z)
• Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks [61.07202852469595]
  We present Neural Splines, a technique for 3D surface reconstruction that is based on random feature kernels arising from infinitely-wide shallow ReLU networks. Our method achieves state-of-the-art results, outperforming recent neural network-based techniques and widely used Poisson Surface Reconstruction.
  arXiv  Detail & Related papers  (2020-06-24T14:54:59Z)
• A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks [87.23360438947114]
  We show that noisy gradient descent with weight decay can still exhibit a " Kernel-like" behavior. This implies that the training loss converges linearly up to a certain accuracy. We also establish a novel generalization error bound for two-layer neural networks trained by noisy gradient descent with weight decay.
  arXiv  Detail & Related papers  (2020-02-10T18:56:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.