Learning Orthonormal Bases for Function Spaces

• URL: http://arxiv.org/abs/2605.19959v1
• Date: Tue, 19 May 2026 15:12:05 GMT
• Title: Learning Orthonormal Bases for Function Spaces
• Abstract summary: We use neural networks to define finite-rank generators of ordinary differential equations (ODEs) governed by skew-adjoint integral operators.<n>We prove a universality result: even with a rank-2 generator, the integrated solutions of the ODE are dense in the group under the appropriate operator topology.
• Score: 10.818158219098562
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Infinite-dimensional orthonormal basis expansions play a central role in representing and computing with function spaces due to their favorable linear algebraic properties. However, common bases such as Fourier or wavelets are fixed and do not adapt to the structure of a given problem or dataset. In this paper, we aim to represent these bases with neural networks and optimize them. Our key idea is that any target infinite-dimensional orthonormal basis can be viewed either as a point on the Lie manifold of the orthogonal group, or equivalently, as the endpoint of a continuous path on that manifold that connects a reference basis, e.g. Fourier, to that target. Paths on the Lie manifold satisfy ordinary differential equations (ODEs) governed by skew-adjoint integral operators. Using neural networks to define finite-rank generators of such ODEs allows us to parameterize and optimize orthonormal bases in function space. While relying on finite-rank generators to model infinite operators might seem restrictive, we prove a universality result: even with a rank-2 generator, the integrated solutions of the ODE are dense in the orthogonal group under the appropriate operator topology. In other words, for any target orthonormal basis, there exists a path originating from a reference basis and driven by finite-rank generators that gets arbitrarily close to that target basis. We demonstrate the flexibility of our framework by transforming the Fourier basis into the principal components of a functional dataset, eigenfunctions of linear operators, or dynamic modes of energy-preserving physical simulations.

Related papers

• Constructing VAE Latent Spaces with Prescribed Topology [5.207307163958805]
  Variational autoencoders learn low-dimensional latent representations of high-dimensional data.<n>Topology-aware models outperform the Gaussian baseline at all practically relevant regularization strengths.
  arXiv  Detail & Related papers  (2026-06-05T08:59:55Z)
• GRIFDIR: Graph Resolution-Invariant FEM Diffusion Models in Function Spaces over Irregular Domains [56.121725064621295]
  Score-based diffusion models in function spaces offer a principled framework for modelling function-valued data.<n>However, practical implementations have struggled to fully realise these benefits.<n>We propose a novel architecture that represents generalised convolutional kernels as finite graph kernels.
  arXiv  Detail & Related papers  (2026-05-05T08:33:52Z)
• Geometric Laplace Neural Operator [12.869633759181417]
  We propose a generalized operator learning framework based on a pole-residue decomposition enriched with exponential basis functions.<n>We introduce the Geometric Laplace Neural Operator (GLNO), which embeds the Laplace spectral representation into the eigen-basis of the Laplace-Beltrami operator.<n>We further design a grid-invariant network architecture (GLNONet) that realizes GLNO in practice.
  arXiv  Detail & Related papers  (2025-12-18T11:07:41Z)
• Learning Eigenstructures of Unstructured Data Manifolds [47.81117132002129]
  We introduce a novel framework that learns a spectral basis for shape and manifold analysis from unstructured data.<n>By replacing the traditional operator selection, construction, and eigendecomposition with a learning-based approach, our framework offers a principled, data-driven alternative to conventional pipelines.
  arXiv  Detail & Related papers  (2025-11-30T22:06:49Z)
• On Counts and Densities of Homogeneous Bent Functions: An Evolutionary Approach [60.00535100780336]
  This paper examines the use of Evolutionary Algorithms (EAs) to evolve homogeneous bent Boolean functions.<n>We introduce the notion of density of homogeneous bent functions, facilitating the algorithmic design that results in finding quadratic and cubic bent functions in different numbers of variables.
  arXiv  Detail & Related papers  (2025-11-16T15:33:40Z)
• Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks [1.5850926890180461]
  Kolmogorov-Arnold Network (KAN) is a network structure recently proposed by Liu et al.<n>Work provides a rigorous theoretical analysis of KAN by establishing generalization bounds for KAN equipped with activation functions.
  arXiv  Detail & Related papers  (2024-10-10T15:23:21Z)
• From Kernel Methods to Neural Networks: A Unifying Variational Formulation [25.6264886382888]
  We present a unifying regularization functional that depends on an operator and on a generic Radon-domain norm. Our framework offers guarantees of universal approximation for a broad family of regularization operators or, equivalently, for a wide variety of shallow neural networks.
  arXiv  Detail & Related papers  (2022-06-29T13:13:53Z)
• Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spaces [52.424621227687894]
  We introduce a unified framework for group equivariant networks on homogeneous spaces. We take advantage of the sparsity of Fourier coefficients of the lifted feature fields. We show that other methods treating features as the Fourier coefficients in the stabilizer subgroup are special cases of our activation.
  arXiv  Detail & Related papers  (2022-06-16T17:59:01Z)
• Generative Adversarial Neural Operators [59.21759531471597]
  We propose the generative adversarial neural operator (GANO), a generative model paradigm for learning probabilities on infinite-dimensional function spaces. GANO consists of two main components, a generator neural operator and a discriminator neural functional. We empirically study GANOs in controlled cases where both input and output functions are samples from GRFs and compare its performance to the finite-dimensional counterpart GAN.
  arXiv  Detail & Related papers  (2022-05-06T05:12:22Z)
• A source fragmentation approach to interacting quantum field theory [0.0]
  We prove that the time-ordered Vacuum Expectation Values and the S-matrix of a regularized Lagrangian quantum theory can be approximated by a local operator. For the Wightman axioms, this suggests a modification that takes the algebra of measurement operators not to be generated by an operator-valued distribution.
  arXiv  Detail & Related papers  (2021-09-09T16:50:43Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.