A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning

• URL: http://arxiv.org/abs/2509.11070v2
• Date: Wed, 17 Sep 2025 03:55:42 GMT
• Title: A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning
• Authors: Jia-Qi Yang, Lei Shi,
• Abstract summary: We develop a framework for approximations between infinite-dimensional spaces using general Mercer operator-valued kernels.<n>Within this framework, we establish dimension-free convergence rates, demonstrating that nonlinear operator learning can overcome the curse of dimensionality.<n>This framework accommodates a wide range of operator learning tasks, ranging from integral operators to architectures based on encoder-decoder representations.
• Score: 7.820614736576814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a stochastic approximation framework for learning nonlinear operators between infinite-dimensional spaces utilizing general Mercer operator-valued kernels. Our framework encompasses two key classes: (i) compact kernels, which admit discrete spectral decompositions, and (ii) diagonal kernels of the form $K(x,x')=k(x,x')T$, where $k$ is a scalar-valued kernel and $T$ is a positive operator on the output space. This broad setting induces expressive vector-valued reproducing kernel Hilbert spaces (RKHSs) that generalize the classical $K=kI$ paradigm, thereby enabling rich structural modeling with rigorous theoretical guarantees. To address target operators lying outside the RKHS, we introduce vector-valued interpolation spaces to precisely quantify misspecification error. Within this framework, we establish dimension-free polynomial convergence rates, demonstrating that nonlinear operator learning can overcome the curse of dimensionality. The use of general operator-valued kernels further allows us to derive rates for intrinsically nonlinear operator learning, going beyond the linear-type behavior inherent in diagonal constructions of $K=kI$. Importantly, this framework accommodates a wide range of operator learning tasks, ranging from integral operators such as Fredholm operators to architectures based on encoder-decoder representations. Moreover, we validate its effectiveness through numerical experiments on the two-dimensional Navier-Stokes equations.

Related papers

• SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels [35.16133249685271]
  We present SVD-NO, a neural operator that parameterizes the kernel by its singular-value decomposition (SVD) and then carries out the integral directly in the low-rank basis.<n>As SVD-NO approximates the full kernel, it obtains a high de- gree of expressivity.
  arXiv  Detail & Related papers  (2025-11-13T07:02:05Z)
• Redefining Neural Operators in $d+1$ Dimensions [16.36239488071517]
  We redefine neural operators on a new $d+1$ dimensional domain based on Schr"odingerisation method.<n>Within this framework, we implement a Schr"odingerised Kernel Neural Operator (SKNO) aligning better with the $d+1$ dimensional evolution.<n>In experiments, the $d+1$ dimensional evolving designs in our SKNO consistently outperform other baselines.
  arXiv  Detail & Related papers  (2025-05-17T00:15:00Z)
• Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning [11.165984223900104]
  This paper introduces the Kernel Neural Operator (KNO), a provably convergent operator-learning architecture.<n>The KNO decouples the choice of kernel from the numerical integration scheme (quadrature)<n> Numerical results demonstrate that on existing benchmarks the training and test accuracy of KNOs is comparable to or higher than popular operator learning techniques.
  arXiv  Detail & Related papers  (2024-06-30T19:28:12Z)
• Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method [16.0198373552099]
  We show how random subspaces can be used to achieve huge computational savings.<n>Our main technical contribution is deriving theoretical guarantees on the effect of the Nystr"om approximation.
  arXiv  Detail & Related papers  (2024-03-05T09:28:40Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Deep Learning with Kernels through RKHM and the Perron-Frobenius Operator [14.877070496733966]
  Reproducing kernel Hilbert $C*$-module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of $C*$-algebra. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators.
  arXiv  Detail & Related papers  (2023-05-23T01:38:41Z)
• Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces [52.35063796758121]
  We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system. We link the risk with the estimation of the spectral decomposition of the Koopman operator. Our results suggest RRR might be beneficial over other widely used estimators.
  arXiv  Detail & Related papers  (2022-05-27T14:57:48Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Neural Operator: Learning Maps Between Function Spaces [75.93843876663128]
  We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations.
  arXiv  Detail & Related papers  (2021-08-19T03:56:49Z)
• Estimating Koopman operators for nonlinear dynamical systems: a nonparametric approach [77.77696851397539]
  The Koopman operator is a mathematical tool that allows for a linear description of non-linear systems. In this paper we capture their core essence as a dual version of the same framework, incorporating them into the Kernel framework. We establish a strong link between kernel methods and Koopman operators, leading to the estimation of the latter through Kernel functions.
  arXiv  Detail & Related papers  (2021-03-25T11:08:26Z)
• High-Dimensional Gaussian Process Inference with Derivatives [90.8033626920884]
  We show that in the low-data regime $ND$, the Gram matrix can be decomposed in a manner that reduces the cost of inference to $mathcalO(N2D + (N2)3)$. We demonstrate this potential in a variety of tasks relevant for machine learning, such as optimization and Hamiltonian Monte Carlo with predictive gradients.
  arXiv  Detail & Related papers  (2021-02-15T13:24:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.