Axiomatizing Neural Networks via Pursuit of Subspaces

• URL: http://arxiv.org/abs/2605.20534v1
• Date: Tue, 19 May 2026 22:12:58 GMT
• Title: Axiomatizing Neural Networks via Pursuit of Subspaces
• Abstract summary: We introduce the Pursuit of Subspaces (PoS) hypothesis, an axiomatic framework that formulates neural network behavior through a set of geometrics.<n>We show that this framework yields geometric explanations for fundamental questions in deep learning, including representation structure, architectural mechanisms, and generalization behavior.
• Score: 18.584665304905897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While deep neural networks have achieved remarkable success across a wide range of domains, their underlying mechanisms remain poorly understood, and they are often regarded as black boxes. This gap between empirical performance and theoretical understanding poses a challenge analogous to the pre-axiomatic stage of classical geometry. In this work, we introduce the Pursuit of Subspaces (PoS) hypothesis, an axiomatic framework that formulates neural network behavior through a set of geometric postulates. These axioms, together with their derived consequences, provide a unified perspective on representation, computation, and generalization in both shallow and deep architectures. We show that this framework yields geometric explanations for fundamental questions in deep learning, including representation structure, architectural mechanisms, and generalization behavior, offering a principled step toward a coherent theoretical foundation.

Related papers

• Pointwise Generalization in Deep Neural Networks [24.565019622696664]
  We identify the structural properties and mathematical principles that explain the tractability of deep networks.<n>We show that deep networks are mathematically tractable in practical regimes and that their generalization is sharply explained by pointwise, feature-spectrum-aware complexity.
  arXiv  Detail & Related papers  (2026-05-18T16:09:06Z)
• Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGI [0.0]
  We argue that artificial general intelligence cannot emerge from current neural network paradigms regardless of scale.<n>We propose a framework distinguishing existential facilities (computational substrate) from architectural organization.
  arXiv  Detail & Related papers  (2025-11-23T16:18:13Z)
• Multi-step Predictive Coding Leads To Simplicity Bias [2.8933605229876664]
  We investigate how the prediction horizon and network depth shape the solutions of predictive coding tasks.<n>We show that sufficiently deep networks trained with multi-step prediction horizons consistently recover the underlying latent structure.<n>Our results provide a principled understanding of when and why predictive coding induces structured representations.
  arXiv  Detail & Related papers  (2025-11-12T12:57:19Z)
• Navigation through Non-Compact Symmetric Spaces: a mathematical perspective on Cartan Neural Networks [0.3749861135832073]
  An initial implementation of these concepts has been presented in a twin paper under the moniker of Cartan Neural Networks.<n>The current paper expands on the mathematical structures underpinning Cartan Neural Networks, detailing the geometric properties of the layers.<n>Together, these papers constitute a first step towards a fully geometrically interpretable theory of neural networks exploiting group-theoretic structures.
  arXiv  Detail & Related papers  (2025-07-22T07:34:53Z)
• The Impact of Geometric Complexity on Neural Collapse in Transfer Learning [6.554326244334867]
  Flatness of the loss surface and neural collapse have recently emerged as useful pre-training metrics.<n>We show through experiments and theory that mechanisms which affect the geometric complexity of the pre-trained network also influence the neural collapse.
  arXiv  Detail & Related papers  (2024-05-24T16:52:09Z)
• Rank Diminishing in Deep Neural Networks [71.03777954670323]
  Rank of neural networks measures information flowing across layers. It is an instance of a key structural condition that applies across broad domains of machine learning. For neural networks, however, the intrinsic mechanism that yields low-rank structures remains vague and unclear.
  arXiv  Detail & Related papers  (2022-06-13T12:03:32Z)
• A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations [77.86290991564829]
  Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. We study a particular sequence of maps between manifold, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between implementing neural networks of practical interest.
  arXiv  Detail & Related papers  (2021-12-17T11:43:30Z)
• Recent advances in deep learning theory [104.01582662336256]
  This paper reviews and organizes the recent advances in deep learning theory. The literature is categorized in six groups: (1) complexity and capacity-based approaches for analysing the generalizability of deep learning; (2) differential equations and their dynamic systems for modelling gradient descent and its variants; (3) the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems; and (5) theoretical foundations of several special structures in network architectures.
  arXiv  Detail & Related papers  (2020-12-20T14:16:41Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)
• A Chain Graph Interpretation of Real-World Neural Networks [58.78692706974121]
  We propose an alternative interpretation that identifies NNs as chain graphs (CGs) and feed-forward as an approximate inference procedure. The CG interpretation specifies the nature of each NN component within the rich theoretical framework of probabilistic graphical models. We demonstrate with concrete examples that the CG interpretation can provide novel theoretical support and insights for various NN techniques.
  arXiv  Detail & Related papers  (2020-06-30T14:46:08Z)
• Understanding Deep Architectures with Reasoning Layer [60.90906477693774]
  We show that properties of the algorithm layers, such as convergence, stability, and sensitivity, are intimately related to the approximation and generalization abilities of the end-to-end model. Our theory can provide useful guidelines for designing deep architectures with reasoning layers.
  arXiv  Detail & Related papers  (2020-06-24T00:26:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.