An Invitation to Neuroalgebraic Geometry

• URL: http://arxiv.org/abs/2501.18915v1
• Date: Fri, 31 Jan 2025 06:33:58 GMT
• Title: An Invitation to Neuroalgebraic Geometry
• Authors: Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta, Matthew Trager, Kathlén Kohn,
• Abstract summary: We promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. We outline a dictionary between algebro-geometric invariants of varieties, such as dimension, degree, and singularities. Work lays the foundations of a research direction bridging algebraic geometry and deep learning.
• Score: 6.369393363312528
• License:
• Abstract: In this expository work, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic models, such as neural networks with polynomial activations, whose associated function spaces are semi-algebraic varieties. We outline a dictionary between algebro-geometric invariants of these varieties, such as dimension, degree, and singularities, and fundamental aspects of machine learning, such as sample complexity, expressivity, training dynamics, and implicit bias. Along the way, we review the literature and discuss ideas beyond the algebraic domain. This work lays the foundations of a research direction bridging algebraic geometry and deep learning, that we refer to as neuroalgebraic geometry.

Related papers

• On the Geometry and Optimization of Polynomial Convolutional Networks [2.9816332334719773]
  We study convolutional neural networks with monomial activation functions. We compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model. For a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
  arXiv  Detail & Related papers  (2024-10-01T14:13:05Z)
• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures [3.5357133304100326]
  Modern machine learning encounters richly structured data that is inherently nonEuclidean. Extracting knowledge from such non-Euclidean data requires a broader mathematical perspective. We propose a graphical taxonomy that integrates recent advances into an intuitive unified framework.
  arXiv  Detail & Related papers  (2024-07-12T17:48:36Z)
• A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on $n$-dimensional Classes [49.32130498861987]
  We study the case of non-differentiable activation functions, such as ReLU. Two recent works introduced a geometric framework to study neural networks. We illustrate our findings with some numerical experiments on classification of images and thermodynamic problems.
  arXiv  Detail & Related papers  (2024-04-09T08:11:46Z)
• A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications [67.33002207179923]
  This paper presents a survey of data structures, models, and applications related to geometric GNNs. We provide a unified view of existing models from the geometric message passing perspective. We also summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation.
  arXiv  Detail & Related papers  (2024-03-01T12:13:04Z)
• Geometry of Polynomial Neural Networks [3.498371632913735]
  We study the expressivity and learning process for neural networks (PNNs) with monomial activation functions. These theoretical results are accompanied by experiments.
  arXiv  Detail & Related papers  (2024-02-01T19:06:06Z)
• A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems [87.30652640973317]
  Recent advances in computational modelling of atomic systems represent them as geometric graphs with atoms embedded as nodes in 3D Euclidean space. Geometric Graph Neural Networks have emerged as the preferred machine learning architecture powering applications ranging from protein structure prediction to molecular simulations and material generation. This paper provides a comprehensive and self-contained overview of the field of Geometric GNNs for 3D atomic systems.
  arXiv  Detail & Related papers  (2023-12-12T18:44:19Z)
• Unraveling the Single Tangent Space Fallacy: An Analysis and Clarification for Applying Riemannian Geometry in Robot Learning [6.253089330116833]
  Handling geometric constraints effectively requires the incorporation of tools from differential geometry into the formulation of machine learning methods. Recent adoption in robot learning has been largely characterized by a mathematically-flawed simplification. This paper provides a theoretical elucidation of various misconceptions surrounding this approach and offers experimental evidence of its shortcomings.
  arXiv  Detail & Related papers  (2023-10-11T21:16:01Z)
• Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles [55.41644538483948]
  We design a deep-learning algorithm for the discovery and identification of the continuous group of symmetries present in a labeled dataset. We use fully connected neural networks to model the transformations symmetry and the corresponding generators. Our study also opens the door for using a machine learning approach in the mathematical study of Lie groups and their properties.
  arXiv  Detail & Related papers  (2023-01-13T16:25:25Z)
• A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations [77.86290991564829]
  Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. We study a particular sequence of maps between manifold, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between implementing neural networks of practical interest.
  arXiv  Detail & Related papers  (2021-12-17T11:43:30Z)
• Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning [109.21780441933164]
  We propose a hybrid approach to improve systematic generalization in reasoning. We showcase a prototype with algebraic representation for the abstract spatial-temporal task of Raven's Progressive Matrices (RPM) We show that the algebraic representation learned can be decoded by isomorphism to generate an answer.
  arXiv  Detail & Related papers  (2021-11-25T09:56:30Z)
• Geometric Algebra Attention Networks for Small Point Clouds [0.0]
  Problems in the physical sciences deal with relatively small sets of points in two- or three-dimensional space. We present rotation- and permutation-equivariant architectures for deep learning on these small point clouds. We demonstrate the usefulness of these architectures by training models to solve sample problems relevant to physics, chemistry, and biology.
  arXiv  Detail & Related papers  (2021-10-05T22:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.