Universality of Shallow and Deep Neural Networks on Non-Euclidean Spaces

• URL: http://arxiv.org/abs/2602.23381v1
• Date: Tue, 03 Feb 2026 17:46:46 GMT
• Title: Universality of Shallow and Deep Neural Networks on Non-Euclidean Spaces
• Abstract summary: We develop a framework for shallow and deep neural networks whose inputs range over a general topological space.<n>We focus on the universal approximation property and establish general conditions under which such networks are dense.<n>In the absence of width constraints, we obtain results that extend classical approximation theorems to non-Euclidean settings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a framework for shallow and deep neural networks whose inputs range over a general topological space. The model is built from a prescribed family of continuous feature maps and a fixed scalar activation function, and it reduces to multilayer feedforward networks in the Euclidean case. We focus on the universal approximation property and establish general conditions under which such networks are dense in spaces of continuous vector-valued functions on arbitrary and locally convex topological spaces. In the absence of width constraints, we obtain universality results that extend classical approximation theorems to non-Euclidean settings. A central focus of the paper is the deep narrow framework, in which the width of each hidden layer is uniformly bounded while the depth is allowed to grow. We identify conditions under which such width constrained deep networks retain universal approximation power. As a concrete example, we employ Ostrand's extension of the Kolmogorov superposition theorem to derive an explicit universality result for products of compact metric spaces, with width bounds expressed in terms of topological dimension.

Related papers

• Across the Loss Landscape with Progressive Growth [51.366966420881646]
  We show how grow-and-optimize strategies to train toward flatter regions by viewing growth as progressive relaxation.<n>We empirically validate these predictions in controlled toy basins and in a realistic ResNet/CI-100 setting.<n>Although progressive subspace growth reliably produces flatter solutions, curvature reductions do not universally translate into improved test performance, highlighting subtleties in the flatness-generalization connection.
  arXiv  Detail & Related papers  (2026-08-25T13:50:48Z)
• The Inductive Bias of Convolutional Neural Networks: Locality and Weight Sharing Reshape Implicit Regularization [57.37943479039033]
  We study how architectural inductive bias reshapes the implicit regularization induced by the edge-of-stability phenomenon in gradient descent.<n>We show that locality and weight sharing fundamentally change this picture.
  arXiv  Detail & Related papers  (2026-03-05T04:50:51Z)
• Dense Neural Networks are not Universal Approximators [53.27010448621372]
  We show that dense neural networks do not possess universality of arbitrary continuous functions.<n>We consider ReLU neural networks subject to natural constraints on weights and input and output dimensions.
  arXiv  Detail & Related papers  (2026-02-07T16:52:38Z)
• On Universality of Deep Equivariant Networks [23.16940006451027]
  Universality results for equivariant neural networks remain rare.<n>We show that with sufficient depth or with the addition of appropriate readout layers, equivariant networks attain universality within the entry-wise separable regime.
  arXiv  Detail & Related papers  (2025-10-17T16:51:31Z)
• On the expressivity of sparse maxout networks [0.6445605125467574]
  We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a maxout activation.<n>We establish a duality between functions computable by such networks and a class of virtual polytopes, linking their geometry to questions of network expressivity.
  arXiv  Detail & Related papers  (2025-10-15T20:18:18Z)
• Convergence of Adam in Deep ReLU Networks via Directional Complexity and Kakeya Bounds [49.1574468325115]
  First-order adaptive optimization methods like Adam are the default choices for training modern deep neural networks.<n>We develop a multi-layer refinement framework that progressively tightens bounds on region crossings.<n>We prove that the number of region crossings collapses from exponential to near-linear in the effective dimension.
  arXiv  Detail & Related papers  (2025-05-21T01:34:16Z)
• On shallow feedforward neural networks with inputs from a topological space [0.0]
  We study feedforward neural networks with inputs from a topological space (TFNNs)<n>We prove a universal approximation theorem for shallow TFNNs, which demonstrates their capacity to approximate any continuous function defined on this topological space.
  arXiv  Detail & Related papers  (2025-04-03T06:48:46Z)
• Data Topology-Dependent Upper Bounds of Neural Network Widths [52.58441144171022]
  We first show that a three-layer neural network can be designed to approximate an indicator function over a compact set. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. We prove the universal approximation property of three-layer ReLU networks using our topological approach.
  arXiv  Detail & Related papers  (2023-05-25T14:17:15Z)
• Quantitative Rates and Fundamental Obstructions to Non-Euclidean Universal Approximation with Deep Narrow Feed-Forward Networks [3.8073142980733]
  We quantify the number of narrow layers required for "deep geometric feed-forward neural networks" We find that both the global and local universal approximation guarantees can only coincide when approximating null-homotopic functions.
  arXiv  Detail & Related papers  (2021-01-13T23:29:40Z)
• Bayesian Deep Ensembles via the Neural Tangent Kernel [49.569912265882124]
  We explore the link between deep ensembles and Gaussian processes (GPs) through the lens of the Neural Tangent Kernel (NTK) We introduce a simple modification to standard deep ensembles training, through addition of a computationally-tractable, randomised and untrainable function to each ensemble member. We prove that our Bayesian deep ensembles make more conservative predictions than standard deep ensembles in the infinite width limit.
  arXiv  Detail & Related papers  (2020-07-11T22:10:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.