Universal Approximation of Nonlinear Operators and Their Derivatives

• URL: http://arxiv.org/abs/2605.15285v1
• Date: Thu, 14 May 2026 18:00:58 GMT
• Title: Universal Approximation of Nonlinear Operators and Their Derivatives
• Abstract summary: Derivative-Informed Operator Learning (DIOL) is an open research frontier at the foundations of the influential field of Operator Learning (OL)<n>We prove the first Universal Approximation Theorems (UATs) of non-linear $k$-times differentiable operators between Banach spaces and their derivatives.<n>Our results are the first complete generalizations of the corresponding influential classical results in [Hornik, 1991] to infinite-dimensional settings and OL.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Derivative-Informed Operator Learning (DIOL), i.e. learning a (nonlinear) operator and its derivatives, is an open research frontier at the foundations of the influential field of Operator Learning (OL). In particular, Universal Approximation Theorems (UATs) of nonlinear operators and their derivatives are foundational open questions and delicate problems in nonlinear functional analysis. In this manuscript, we prove the first UATs of non-linear $k$-times differentiable operators between Banach spaces and their derivatives, uniformly on compact sets and in weighted Sobolev norms for general finite input measures, via OL architectures. Our results are the first complete generalizations of the corresponding influential classical results in [Hornik, 1991] to infinite-dimensional settings and OL. We discuss several open areas where DIOL and our UATs find applications: high-order accuracy in OL, fast constrained optimization in Banach spaces (e.g. optimal control of PDEs, inverse problems) and numerical methods for infinite-dimensional PDEs (e.g. HJB PDEs on Banach spaces from optimal control of PDEs, SPDEs, path-dependent systems, partially observed systems, mean-field control). We parameterize nonlinear operators via Encoder-Decoder Architectures, renowned classes in OL due to their generality, including classical architectures, such as DeepONets, Deep-H-ONets, PCA-Nets. Our results are based on four key features that allow us to prove UATs in full generality: (i) Approximation Properties of Banach spaces. (ii) $k$-times continuous differentiability in the sense of Bastiani (weaker than $k$-times continuous Fréchet differentiability). (iii) Natural compact-open topologies for UA; indeed, we show that UA in standard compact-open topologies induced by operator norms is violated even for Fréchet derivatives. (iv) Construction of novel weighted Sobolev spaces for the UA.

Related papers

• GRIFDIR: Graph Resolution-Invariant FEM Diffusion Models in Function Spaces over Irregular Domains [56.121725064621295]
  Score-based diffusion models in function spaces offer a principled framework for modelling function-valued data.<n>However, practical implementations have struggled to fully realise these benefits.<n>We propose a novel architecture that represents generalised convolutional kernels as finite graph kernels.
  arXiv  Detail & Related papers  (2026-05-05T08:33:52Z)
• Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control [2.421910312262742]
  We develop deep learning-based approximation methods for fully nonlinear second-order PDEs on separable Hilbert spaces, such as HJB equations for infinite-dimensional control, by parameterizing solutions via Hilbert--Galerkin Neural Operators (HGNOs)<n>We prove the first Universal Approximation Theorems (UATs) which are sufficiently powerful to address these problems, based on novel topologies for Hessian terms and corresponding novel continuity assumptions on the fully nonlinear operator.
  arXiv  Detail & Related papers  (2026-03-19T20:53:20Z)
• Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds [63.47271262149291]
  We propose a unified framework for PAC-Bayesian norm-based generalization.<n>The key to our approach is a sensitivity matrix that quantifies the network outputs with respect to structured weight perturbations.<n>We derive a family of generalization bounds that recover several existing PAC-Bayesian results as special cases.
  arXiv  Detail & Related papers  (2026-01-13T00:42:22Z)
• From Product Hilbert Spaces to the Generalized Koopman Operator and the Nonlinear Fundamental Lemma [0.0]
  We prove that there exists an infinite-dimensional linear operator, i.e. the generalized Koopman operator.<n>A scalable data-driven method for computing finite-dimensional approximations of generalized Koopman operators is presented.<n>We derive a nonlinear fundamental lemma by exploiting the bilinear structure of the infinite-dimensional generalized Koopman model.
  arXiv  Detail & Related papers  (2025-08-10T21:57:16Z)
• Convergence of TD(0) under Polynomial Mixing with Nonlinear Function Approximation [49.1574468325115]
  Temporal Difference Learning (TD(0)) is fundamental in reinforcement learning.<n>We provide the first high-probability, finite-sample analysis of vanilla TD(0) on mixing Markov data.
  arXiv  Detail & Related papers  (2025-02-08T22:01:02Z)
• DimINO: Dimension-Informed Neural Operator Learning [41.37905663176428]
  DimINO is a framework inspired by dimensional analysis.<n>It can be seamlessly integrated into existing neural operator architectures.<n>It achieves up to 76.3% performance gain on PDE datasets.
  arXiv  Detail & Related papers  (2024-10-08T10:48:50Z)
• Optimal deep learning of holomorphic operators between Banach spaces [0.6554326244334866]
  We tackle the problem of learning operators between Banach spaces, in contrast to the vast majority of past works considering only Hilbert spaces. We combine arbitrary approximate encoders and decoders with standard feedforward Deep Neural Network (DNN) architectures. We show that DL is optimal for this problem: no recovery procedure can surpass these generalization bounds up to log terms.
  arXiv  Detail & Related papers  (2024-06-20T01:49:42Z)
• Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities [26.039367753810705]
  We prove a representer theorem that states that the solution sets to learning problems posed over Banach spaces are completely characterized by neural architectures with nonlinearities.<n>Our results shed light on the regularity of functions learned by neural networks trained on data, and provide new theoretical motivation for several architectural choices found in practice.
  arXiv  Detail & Related papers  (2023-10-05T17:13:16Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Neural and spectral operator surrogates: unified construction and expression rate bounds [0.46040036610482665]
  We study approximation rates for deep surrogates of maps between infinite-dimensional function spaces. Operator in- and outputs from function spaces are assumed to be parametrized by stable, affine representation systems.
  arXiv  Detail & Related papers  (2022-07-11T15:35:14Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)
• Neural Operator: Graph Kernel Network for Partial Differential Equations [57.90284928158383]
  This work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators) We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
  arXiv  Detail & Related papers  (2020-03-07T01:56:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.