Emergent Riemannian geometry over learning discrete computations on continuous manifolds

• URL: http://arxiv.org/abs/2512.00196v1
• Date: Fri, 28 Nov 2025 20:29:06 GMT
• Title: Emergent Riemannian geometry over learning discrete computations on continuous manifolds
• Authors: Julian Brandon, Angus Chadwick, Arthur Pellegrino,
• Abstract summary: We show that signatures of discrete computations emerge in the representational geometry of neural networks as they learn.<n>We demonstrate how different learning regimes (rich vs. lazy) have contrasting metric and curvature structures, affecting the ability of the networks to generalise to unseen inputs.
• Score: 1.8665975431697432
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many tasks require mapping continuous input data (e.g. images) to discrete task outputs (e.g. class labels). Yet, how neural networks learn to perform such discrete computations on continuous data manifolds remains poorly understood. Here, we show that signatures of such computations emerge in the representational geometry of neural networks as they learn. By analysing the Riemannian pullback metric across layers of a neural network, we find that network computation can be decomposed into two functions: discretising continuous input features and performing logical operations on these discretised variables. Furthermore, we demonstrate how different learning regimes (rich vs. lazy) have contrasting metric and curvature structures, affecting the ability of the networks to generalise to unseen inputs. Overall, our work provides a geometric framework for understanding how neural networks learn to perform discrete computations on continuous manifolds.

Related papers

• RNNs perform task computations by dynamically warping neural representations [2.2917707112773598]
  We show that recurrent neural networks (RNNs) perform computations by dynamically warping their representations of task variables.<n>By characterising the time-varying geometry of RNNs, we show that dynamic warping is a fundamental feature of their computations.
  arXiv  Detail & Related papers  (2025-12-03T23:01:43Z)
• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Riemannian Residual Neural Networks [58.925132597945634]
  We show how to extend the residual neural network (ResNet) ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks.
  arXiv  Detail & Related papers  (2023-10-16T02:12:32Z)
• Predictions Based on Pixel Data: Insights from PDEs and Finite Differences [0.0]
  This paper deals with approximation of time sequences where each observation is a matrix. We show that with relatively small networks, we can represent exactly a class of numerical discretizations of PDEs based on the method of lines. Our network architecture is inspired by those typically adopted in the approximation of time sequences.
  arXiv  Detail & Related papers  (2023-05-01T08:54:45Z)
• How does training shape the Riemannian geometry of neural network representations? [36.26020698558347]
  We study how training shapes the geometry induced by unconstrained neural network feature maps.<n>We first show that at infinite width, neural networks with random parameters induce highly symmetric metrics on input space.<n>This symmetry is broken by feature learning: networks trained to perform classification tasks learn to magnify local areas along decision boundaries.
  arXiv  Detail & Related papers  (2023-01-26T19:43:16Z)
• Discretization Invariant Networks for Learning Maps between Neural Fields [3.09125960098955]
  We present a new framework for understanding and designing discretization invariant neural networks (DI-Nets) Our analysis establishes upper bounds on the deviation in model outputs under different finite discretizations. We prove by construction that DI-Nets universally approximate a large class of maps between integrable function spaces.
  arXiv  Detail & Related papers  (2022-06-02T17:44:03Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Deep Parametric Continuous Convolutional Neural Networks [92.87547731907176]
  Parametric Continuous Convolution is a new learnable operator that operates over non-grid structured data. Our experiments show significant improvement over the state-of-the-art in point cloud segmentation of indoor and outdoor scenes.
  arXiv  Detail & Related papers  (2021-01-17T18:28:23Z)
• Continuous-in-Depth Neural Networks [107.47887213490134]
  We first show that ResNets fail to be meaningful dynamical in this richer sense. We then demonstrate that neural network models can learn to represent continuous dynamical systems. We introduce ContinuousNet as a continuous-in-depth generalization of ResNet architectures.
  arXiv  Detail & Related papers  (2020-08-05T22:54:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.