Operator-Theoretic Generalization Bounds for Multitask Deep Learning

• URL: http://arxiv.org/abs/2608.15982v1
• Date: Mon, 17 Aug 2026 00:36:57 GMT
• Title: Operator-Theoretic Generalization Bounds for Multitask Deep Learning
• Abstract summary: We develop operator-theoretic generalization bounds for deep multi-output function classes.<n>We derive Rademacher bounds complexity for invertible and width-expanding injective architectures.
• Score: 0.0254890465057467
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop operator-theoretic generalization bounds for deep multi-output function classes by representing network layers as Koopman composition operators on vector-valued reproducing kernel Hilbert spaces. In vector-valued Sobolev RKHSs, we derive Rademacher complexity bounds for invertible and width-expanding injective architectures. The estimates separate the output-coupling contribution, represented by the trace of the task matrix, from the layerwise operator norms, Sobolev symbol ratios, determinant factors, and restriction constants generated by the linear maps. We then analyze a distinct one-dimensional Brownian/Cameron--Martin regime. Using the exact anchored derivative-norm characterization of the vector-valued Brownian RKHS, we obtain layerwise bounds for domain-preserving scalar linear maps and anchored diffeomorphic activations; the corresponding factors scale as $|W_l|^{1/2}$ and $\|σ_l'\|_\infty^{1/2}$, respectively, and do not involve Sobolev smoothness exponents. Because the Sobolev and Brownian results concern different hypothesis spaces, neither is asserted to dominate the other uniformly. We additionally formulate shared operator learning across tasks, prove a finite-rank representer theorem, derive the exact finite-dimensional problem for squared loss, and establish a target-transfer bound when the learned operator is obtained independently of the target sample. Synthetic and MNIST studies examine stabilized Sobolev-inspired and Brownian-inspired complexity proxies; these empirical proxies are not evaluations of the proved bounds for rank-deficient architectures.

Related papers

• RKHS Representation of Algebraic Convolutional Filters with Integral Operators [111.57971404925486]
  In this paper, we develop a theory showing that the range of integral operators naturally induces RKHS convolutional signal models.<n>We show that filtering with integral operators corresponds to iterated box products, giving rise to a unital kernel algebra.<n>Our results establish precise connections between eigendecompositions and RKHS representations in graphon signal processing, extend naturally to directed graphons, and enable novel spatial-spectral localization results.
  arXiv  Detail & Related papers  (2026-02-22T08:28:34Z)
• Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori Certification [45.85698554568285]
  We develop a deterministic zeroth-order mirror descent framework by replacing gradients with a general vector field.<n>Our analysis provides an evaluation template for last-rate inequality evaluations.<n>Together, these results expose a hidden geometric linking Bregman identities, deterministic certification, and robust conic geometry in zeroth-order mirror descent.
  arXiv  Detail & Related papers  (2026-01-31T10:05:05Z)
• Deep Delta Learning [91.75868893250662]
  We introduce Deep Delta Learning (DDL), a novel architecture that generalizes the standard residual connection.<n>We provide a spectral analysis of this operator, demonstrating that the gate $(mathbfX)$ enables dynamic between identity mapping, projection, and geometric reflection.<n>This unification empowers the network to explicitly control the spectrum of its layer-wise transition operator, enabling the modeling of complex, non-monotonic dynamics.
  arXiv  Detail & Related papers  (2026-01-01T18:11:38Z)
• Deep Eigenspace Network and Its Application to Parametric Non-selfadjoint Eigenvalue Problems [0.12744523252873352]
  We consider operator learning for efficiently solving parametric non-selfadjoint eigenvalue problems.<n>We introduce a hybrid framework that learns the stable invariant eigensubspace mapping rather than individual eigenfunctions.
  arXiv  Detail & Related papers  (2025-12-23T05:20:22Z)
• A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning [7.820614736576814]
  We develop a framework for approximations between infinite-dimensional spaces using general Mercer operator-valued kernels.<n>Within this framework, we establish dimension-free convergence rates, demonstrating that nonlinear operator learning can overcome the curse of dimensionality.<n>This framework accommodates a wide range of operator learning tasks, ranging from integral operators to architectures based on encoder-decoder representations.
  arXiv  Detail & Related papers  (2025-09-14T03:33:36Z)
• Self-Attention as a Parametric Endofunctor: A Categorical Framework for Transformer Architectures [0.0]
  We develop a category-theoretic framework focusing on the linear components of self-attention.<n>We show that the query, key, and value maps naturally define a parametric 1-morphism in the 2-category $mathbfPara(Vect)$.<n> stacking multiple self-attention layers corresponds to constructing the free monad on this endofunctor.
  arXiv  Detail & Related papers  (2025-01-06T11:14:18Z)
• Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks [1.5850926890180461]
  Kolmogorov-Arnold Network (KAN) is a network structure recently proposed by Liu et al.<n>Work provides a rigorous theoretical analysis of KAN by establishing generalization bounds for KAN equipped with activation functions.
  arXiv  Detail & Related papers  (2024-10-10T15:23:21Z)
• Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
  Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.<n>This paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective.
  arXiv  Detail & Related papers  (2024-07-15T07:11:44Z)
• Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces [52.35063796758121]
  We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system. We link the risk with the estimation of the spectral decomposition of the Koopman operator. Our results suggest RRR might be beneficial over other widely used estimators.
  arXiv  Detail & Related papers  (2022-05-27T14:57:48Z)
• Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
  We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces. We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
  arXiv  Detail & Related papers  (2022-04-20T21:15:38Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems [107.3868459697569]
  We introduce an eigendecomposition-free approach to training a deep network. We show that our approach is much more robust than explicit differentiation of the eigendecomposition. Our method has better convergence properties and yields state-of-the-art results.
  arXiv  Detail & Related papers  (2020-04-15T04:29:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.