A General Method for Proving Networks Universal Approximation Property

• URL: http://arxiv.org/abs/2511.07857v1
• Date: Wed, 12 Nov 2025 01:24:24 GMT
• Title: A General Method for Proving Networks Universal Approximation Property
• Abstract summary: This paper proposes a general and modular framework for proving universal approximation.<n>We show that any deep network composed of such modules inherently retains the universal approximation property.
• Score: 3.565474315629056
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning architectures are highly diverse. To prove their universal approximation properties, existing works typically rely on model-specific proofs. Generally, they construct a dedicated mathematical formulation for each architecture (e.g., fully connected networks, CNNs, or Transformers) and then prove their universal approximability. However, this approach suffers from two major limitations: first, every newly proposed architecture often requires a completely new proof from scratch; second, these proofs are largely isolated from one another, lacking a common analytical foundation. This not only incurs significant redundancy but also hinders unified theoretical understanding across different network families. To address these issues, this paper proposes a general and modular framework for proving universal approximation. We define a basic building block (comprising one or multiple layers) that possesses the universal approximation property as a Universal Approximation Module (UAM). Under this condition, we show that any deep network composed of such modules inherently retains the universal approximation property. Moreover, the overall approximation process can be interpreted as a progressive refinement across modules. This perspective not only unifies the analysis of diverse architectures but also enables a step-by-step understanding of how expressive power evolves through the network.

Related papers

• Beyond $L_2$: Generalizing Abductive Latent Explanations to Diverse Prototype-Based Architectures [7.001970497421476]
  Abductive Latent Explanations (ALE) were introduced to provide formal, mathematically guaranteed explanations.<n>ALEs rely on computing tight bounds on latent space distances to produce formal explanations.<n>In this work, we generalize the ALE framework to support non-Euclidean prototype architectures.
  arXiv  Detail & Related papers  (2026-08-17T16:20:20Z)
• Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection [58.25418970608328]
  We study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains.<n>Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution.<n>We propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem.
  arXiv  Detail & Related papers  (2026-07-08T09:27:15Z)
• Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime [24.053364183688874]
  Training dynamics is central to understanding neural networks.<n>We propose a convergence framework for analyzing gradient descent dynamics under a broad family of neural network architectures.
  arXiv  Detail & Related papers  (2026-06-22T14:00:26Z)
• Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities [52.189426539304065]
  This paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization"<n>We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge.
  arXiv  Detail & Related papers  (2026-04-09T03:35:21Z)
• Neural Network Approximation: A View from Polytope Decomposition [36.24385738607139]
  We develop an explicit kernel method to derive an universal approximation of continuous functions.<n>A ReLU network is constructed to approximate the kernel in each subdomain separately.<n>We extend our approach to analytic functions to reach a higher approximation rate.
  arXiv  Detail & Related papers  (2026-01-26T08:39:11Z)
• On Universality of Deep Equivariant Networks [23.16940006451027]
  Universality results for equivariant neural networks remain rare.<n>We show that with sufficient depth or with the addition of appropriate readout layers, equivariant networks attain universality within the entry-wise separable regime.
  arXiv  Detail & Related papers  (2025-10-17T16:51:31Z)
• On Universality Classes of Equivariant Networks [9.137637807153464]
  We investigate the approximation power of equivariant neural networks beyond separation constraints.<n>We show that separation power does not fully capture expressivity.<n>We identify settings where shallow equivariant networks do achieve universality.
  arXiv  Detail & Related papers  (2025-06-02T22:07:52Z)
• Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines [15.67299102925013]
  We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps.<n>Our main theorem also unifies the universal approximation theorems for both shallow and deep networks.
  arXiv  Detail & Related papers  (2024-05-22T14:25:02Z)
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration [62.41329042683779]
  We propose a high-accuracy rotation equivariant proximal network that embeds rotation symmetry priors into the deep unfolding framework. This study makes efforts to suggest a high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework.
  arXiv  Detail & Related papers  (2023-12-25T11:53:06Z)
• Monotone deep Boltzmann machines [86.50247625239406]
  Deep Boltzmann machines (DBMs) are multi-layered probabilistic models governed by a pairwise energy function. We develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer. We show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution.
  arXiv  Detail & Related papers  (2023-07-11T03:02:44Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Generalization by design: Shortcuts to Generalization in Deep Learning [7.751691910877239]
  We show that good generalization may be instigated by bounded spectral products over layers leading to a novel geometric regularizer. Backed up by theory we further demonstrate that "generalization by design" is practically possible and that good generalization may be encoded into the structure of the network.
  arXiv  Detail & Related papers  (2021-07-05T20:01:23Z)
• Polynomial Networks in Deep Classifiers [55.90321402256631]
  We cast the study of deep neural networks under a unifying framework. Our framework provides insights on the inductive biases of each model. The efficacy of the proposed models is evaluated on standard image and audio classification benchmarks.
  arXiv  Detail & Related papers  (2021-04-16T06:41:20Z)
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators [72.62940905965267]
  Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. Are CF-INNs universal approximators for invertible functions? We prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes.
  arXiv  Detail & Related papers  (2020-06-20T02:07:37Z)
• Non-Euclidean Universal Approximation [4.18804572788063]
  Modifications to a neural network's input and output layers are often required to accommodate the specificities of most practical learning tasks. We present general conditions describing feature and readout maps that preserve an architecture's ability to approximate any continuous functions uniformly on compacts.
  arXiv  Detail & Related papers  (2020-06-03T15:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.