From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons

• URL: http://arxiv.org/abs/2510.15012v1
• Date: Thu, 16 Oct 2025 13:15:39 GMT
• Title: From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons
• Authors: Yi-Shan Chu, Yueh-Cheng Kuo,
• Abstract summary: We revisit the Universal Approximation Theorem through the lens of the tropical geometry of neural networks.<n>We introduce constructive, geometry-aware architecture for sigmoidal multi-layer perceptrons.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We revisit the Universal Approximation Theorem(UAT) through the lens of the tropical geometry of neural networks and introduce a constructive, geometry-aware initialization for sigmoidal multi-layer perceptrons (MLPs). Tropical geometry shows that Rectified Linear Unit (ReLU) networks admit decision functions with a combinatorial structure often described as a tropical rational, namely a difference of tropical polynomials. Focusing on planar binary classification, we design purely sigmoidal MLPs that adhere to the finite-sum format of UAT: a finite linear combination of shifted and scaled sigmoids of affine functions. The resulting models yield decision boundaries that already align with prescribed shapes at initialization and can be refined by standard training if desired. This provides a practical bridge between the tropical perspective and smooth MLPs, enabling interpretable, shape-driven initialization without resorting to ReLU architectures. We focus on the construction and empirical demonstrations in two dimensions; theoretical analysis and higher-dimensional extensions are left for future work.

Related papers

• Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry [21.251058776601553]
  We present the first analysis of MoE through the lens of tropical geometry.<n>Our framework unifies the discrete geometry of the Hyperx with the continuous geometry of neural functions.
  arXiv  Detail & Related papers  (2026-02-03T07:17:38Z)
• Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds [0.5729426778193398]
  We study the emergence of multi-step reasoning in deep Transformer language models through a geometric and statistical-physics lens.<n>We formalize the forward pass as a discrete coarse-graining map and relate the appearance of stable "concept basins" to fixed points of this renormalization-like dynamics.<n>The resulting low-entropy regime is characterized by a spectral tail collapse and by the formation of transient, reusable object-like structures in representation space.
  arXiv  Detail & Related papers  (2026-01-16T23:11:02Z)
• FoundationSLAM: Unleashing the Power of Depth Foundation Models for End-to-End Dense Visual SLAM [50.9765003472032]
  FoundationSLAM is a learning-based monocular dense SLAM system for accurate and robust tracking and mapping.<n>Our core idea is to bridge flow estimation with reasoning by leveraging the guidance from foundation depth models.
  arXiv  Detail & Related papers  (2025-12-31T17:57:45Z)
• The Neural Differential Manifold: An Architecture with Explicit Geometric Structure [8.201374511929538]
  This paper introduces the Neural Differential Manifold (NDM), a novel neural network architecture that explicitly incorporates geometric structure into its fundamental design.<n>We analyze the theoretical advantages of this approach, including its potential for more efficient optimization, enhanced continual learning, and applications in scientific discovery and controllable generative modeling.
  arXiv  Detail & Related papers  (2025-10-29T02:24:27Z)
• GeoFunFlow: Geometric Function Flow Matching for Inverse Operator Learning over Complex Geometries [7.205211713278516]
  Inverse problems governed by partial differential equations (PDEs) are crucial in science and engineering.<n>We introduce em GeoFunFlow, a geometric diffusion model framework for inverse problems on complex geometries.
  arXiv  Detail & Related papers  (2025-09-28T23:21:52Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• Tropical Geometry Based Edge Detection Using Min-Plus and Max-Plus Algebra [0.0]
  This paper proposes a tropical geometry-based edge detection framework that reformulates convolution and gradient computations using min-plus and max-plus algebra.<n>The tropical formulation emphasizes dominant intensity variations, contributing to sharper and more continuous edge representations.
  arXiv  Detail & Related papers  (2025-05-24T10:19:27Z)
• The Real Tropical Geometry of Neural Networks [15.07926301607672]
  We study the classification of ReLU neural networks as a tropical rational function. Our findings extend and refine the connection between neural networks and tropical geometry by observing structures established in real tropical geometry.
  arXiv  Detail & Related papers  (2024-03-18T15:24:47Z)
• A Theory of Topological Derivatives for Inverse Rendering of Geometry [87.49881303178061]
  We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives. We validate the proposed theory with optimization of closed curves in 2D and surfaces in 3D to lend insights into limitations of current methods.
  arXiv  Detail & Related papers  (2023-08-19T00:55:55Z)
• Geometric Clifford Algebra Networks [53.456211342585824]
  We propose Geometric Clifford Algebra Networks (GCANs) for modeling dynamical systems. GCANs are based on symmetry group transformations using geometric (Clifford) algebras.
  arXiv  Detail & Related papers  (2023-02-13T18:48:33Z)
• Dist2Cycle: A Simplicial Neural Network for Homology Localization [66.15805004725809]
  Simplicial complexes can be viewed as high dimensional generalizations of graphs that explicitly encode multi-way ordered relations. We propose a graph convolutional model for learning functions parametrized by the $k$-homological features of simplicial complexes.
  arXiv  Detail & Related papers  (2021-10-28T14:59:41Z)
• ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
  In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
  arXiv  Detail & Related papers  (2021-02-16T04:07:13Z)
• Shape Estimation for Elongated Deformable Object using B-spline Chained Multiple Random Matrices Model [5.94069939063928]
  A B-spline chained multiple random matrices representation is proposed to model geometric characteristics of an elongated deformable object. An expectation-maximization (EM) method is derived to estimate the shape of the elongated deformable object. The proposed algorithm is evaluated for the shape estimation of the elongated deformable objects in scenarios.
  arXiv  Detail & Related papers  (2020-04-10T21:15:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.