The Real Tropical Geometry of Neural Networks

• URL: http://arxiv.org/abs/2403.11871v1
• Date: Mon, 18 Mar 2024 15:24:47 GMT
• Title: The Real Tropical Geometry of Neural Networks
• Authors: Marie-Charlotte Brandenburg, Georg Loho, Guido Montúfar,
• Abstract summary: We study the classification of ReLU neural networks as a tropical rational function. Our findings extend and refine the connection between neural networks and tropical geometry by observing structures established in real tropical geometry.
• Score: 15.07926301607672
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider a binary classifier defined as the sign of a tropical rational function, that is, as the difference of two convex piecewise linear functions. The parameter space of ReLU neural networks is contained as a semialgebraic set inside the parameter space of tropical rational functions. We initiate the study of two different subdivisions of this parameter space: a subdivision into semialgebraic sets, on which the combinatorial type of the decision boundary is fixed, and a subdivision into a polyhedral fan, capturing the combinatorics of the partitions of the dataset. The sublevel sets of the 0/1-loss function arise as subfans of this classification fan, and we show that the level-sets are not necessarily connected. We describe the classification fan i) geometrically, as normal fan of the activation polytope, and ii) combinatorially through a list of properties of associated bipartite graphs, in analogy to covector axioms of oriented matroids and tropical oriented matroids. Our findings extend and refine the connection between neural networks and tropical geometry by observing structures established in real tropical geometry, such as positive tropicalizations of hypersurfaces and tropical semialgebraic sets.

Related papers

• From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons [0.0]
  We revisit the Universal Approximation Theorem through the lens of the tropical geometry of neural networks.<n>We introduce constructive, geometry-aware architecture for sigmoidal multi-layer perceptrons.
  arXiv  Detail & Related papers  (2025-10-16T13:15:39Z)
• Compositional Symmetry as Compression: Lie Pseudogroup Structure in Algorithmic Agents [0.0]
  In the (Kolmogorov) view, agents are programs that track and compress sensory streams using generative programs.<n>We propose a framework where the relevant structural prior is simplicity (off) understood as emphSolomonal symmetry<n>We show that accurate world-tracking imposes (i) emphdynamic constraints -- and (ii) emphdynamic constraints under static inputs.
  arXiv  Detail & Related papers  (2025-10-12T13:06:37Z)
• Tessellation Groups, Harmonic Analysis on Non-compact Symmetric Spaces and the Heat Kernel in view of Cartan Convolutional Neural Networks [0.0]
  This paper focuses on some mathematical foundational aspects that we deem necessary for our next steps forward.<n>The aim is to introduce layers that are mathematically modeled as non-compact symmetric spaces.<n>In particular, we have introduced the notion of Tits Satake (TS) vector bundles where the TS submanifold is the base space.
  arXiv  Detail & Related papers  (2025-08-22T00:27:03Z)
• A Set-to-Set Distance Measure in Hyperbolic Space [50.134086375286074]
  We propose a hyperbolic set-to-set distance measure for computing dissimilarity between sets in hyperbolic space.<n>By considering the topological differences, HS2SD provides a more nuanced understanding of the relationships between two hyperbolic sets.
  arXiv  Detail & Related papers  (2025-06-23T11:31:40Z)
• How to Learn a Star: Binary Classification with Starshaped Polyhedral Sets [1.360022695699485]
  We consider binary classification restricted to a class of continuous piecewise functions whose decision boundaries are (possibly non) star polyhedral sets.<n>We investigate the geometric structure of the loss-functions most prominently the sublevel sets for two loss-functions: 0/1-loss (discrete loss) and an exponential loss function.
  arXiv  Detail & Related papers  (2025-05-02T15:33:31Z)
• A Relative Homology Theory of Representation in Neural Networks [0.0]
  Previous research has proven that the set of maps implemented by neural networks with a ReLU activation function is identical to the set of piecewise linear continuous maps. We leverage these properties to define the equivalence class of inputs $sim_Phi$, which can be split into two sets related to the local rank of $Phi_J$ and the intersections $cap textImPhi_J_i$. We refer to the latter as the overlap decomposition $O_Phi$ and prove that if the intersections between each polyhedron and the input manifold are
  arXiv  Detail & Related papers  (2025-02-03T13:52:17Z)
• Relative Representations: Topological and Geometric Perspectives [53.88896255693922]
  Relative representations are an established approach to zero-shot model stitching. We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations. Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• Low-dimensional polaritonics: Emergent non-trivial topology on exciton-polariton simulators [0.0]
  Polaritonic lattice configurations in dimensions $D=2$ are used as simulators of topological phases, based on symmetry class A Hamiltonians. We provide a comprehensive mathematical framework, which fully addresses the source and structure of topological phases in coupled polaritonic array systems.
  arXiv  Detail & Related papers  (2023-10-31T04:22:58Z)
• Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks [40.137069931650444]
  Tropical geometry has recently found several applications in the analysis of neural networks with piecewise linear activation functions. This paper presents a new look at the problem of tropical division and its application to the simplification of neural networks.
  arXiv  Detail & Related papers  (2023-06-27T02:26:07Z)
• Data Topology-Dependent Upper Bounds of Neural Network Widths [52.58441144171022]
  We first show that a three-layer neural network can be designed to approximate an indicator function over a compact set. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. We prove the universal approximation property of three-layer ReLU networks using our topological approach.
  arXiv  Detail & Related papers  (2023-05-25T14:17:15Z)
• Mathematical Foundations for a Compositional Account of the Bayesian Brain [0.0]
  We use the tools of contemporary applied category theory to supply functorial semantics for approximate inference. We define fibrations of statistical games and classify various problems of statistical inference as corresponding sections. We construct functors which explain the compositional structure of predictive coding neural circuits under the free energy principle.
  arXiv  Detail & Related papers  (2022-12-23T18:58:17Z)
• Spectral embedding and the latent geometry of multipartite networks [67.56499794542228]
  Many networks are multipartite, meaning their nodes can be divided into partitions and nodes of the same partition are never connected. This paper demonstrates that the node representations obtained via spectral embedding live near partition-specific low-dimensional subspaces of a higher-dimensional ambient space. We propose a follow-on step after spectral embedding, to recover node representations in their intrinsic rather than ambient dimension.
  arXiv  Detail & Related papers  (2022-02-08T15:52:03Z)
• Dist2Cycle: A Simplicial Neural Network for Homology Localization [66.15805004725809]
  Simplicial complexes can be viewed as high dimensional generalizations of graphs that explicitly encode multi-way ordered relations. We propose a graph convolutional model for learning functions parametrized by the $k$-homological features of simplicial complexes.
  arXiv  Detail & Related papers  (2021-10-28T14:59:41Z)
• Topological aspects of the critical three-state Potts model [0.0]
  A complete characterization is obtained by breaking down the Fuchs-Runkel-Schweigert construction of 2d rational CFT to the lattice setting. The symmetries are represented by matrix product operators (MPO), as well as intertwiners between the diagonal tetracritical Ising model and the non-diagonal three-state Potts model.
  arXiv  Detail & Related papers  (2021-07-22T16:43:40Z)
• Quantum critical phase transition between two topologically-ordered phases in the Ising toric code bilayer [0.0]
  We show that two toric code layers on the square lattice coupled by an Ising interaction display two distinct phases with intrinsic topological order. The second-order quantum phase transition between the weakly-coupled $mathbbZtimesmathbbZ$ and the strongly-coupled $mathbbZ$ can be described by the condensation of bosonic quasiparticles from both sides.
  arXiv  Detail & Related papers  (2020-10-12T19:16:36Z)
• Dynamical solitons and boson fractionalization in cold-atom topological insulators [110.83289076967895]
  We study the $mathbbZ$ Bose-Hubbard model at incommensurate densities. We show how defects in the $mathbbZ$ field can appear in the ground state, connecting different sectors. Using a pumping argument, we show that it survives also for finite interactions.
  arXiv  Detail & Related papers  (2020-03-24T17:31:34Z)
• On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective [54.1171355815052]
  This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, ReLU, Affine)
  arXiv  Detail & Related papers  (2020-02-20T16:22:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.