Algebraically-Informed Deep Networks (AIDN): A Deep Learning Approach to Represent Algebraic Structures

• URL: http://arxiv.org/abs/2012.01141v3
• Date: Fri, 12 Feb 2021 07:06:52 GMT
• Title: Algebraically-Informed Deep Networks (AIDN): A Deep Learning Approach to Represent Algebraic Structures
• Authors: Mustafa Hajij, Ghada Zamzmi, Matthew Dawson, Greg Muller
• Abstract summary: We introduce textbfAIDN, textitAlgebraically-Informed Deep Networks. textbfAIDN is a deep learning algorithm to represent any finitely-presented algebraic object with a set of deep neural networks.
• Score: 0.688204255655161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One of the central problems in the interface of deep learning and mathematics is that of building learning systems that can automatically uncover underlying mathematical laws from observed data. In this work, we make one step towards building a bridge between algebraic structures and deep learning, and introduce \textbf{AIDN}, \textit{Algebraically-Informed Deep Networks}. \textbf{AIDN} is a deep learning algorithm to represent any finitely-presented algebraic object with a set of deep neural networks. The deep networks obtained via \textbf{AIDN} are \textit{algebraically-informed} in the sense that they satisfy the algebraic relations of the presentation of the algebraic structure that serves as the input to the algorithm. Our proposed network can robustly compute linear and non-linear representations of most finitely-presented algebraic structures such as groups, associative algebras, and Lie algebras. We evaluate our proposed approach and demonstrate its applicability to algebraic and geometric objects that are significant in low-dimensional topology. In particular, we study solutions for the Yang-Baxter equations and their applications on braid groups. Further, we study the representations of the Temperley-Lieb algebra. Finally, we show, using the Reshetikhin-Turaev construction, how our proposed deep learning approach can be utilized to construct new link invariants. We believe the proposed approach would tread a path toward a promising future research in deep learning applied to algebraic and geometric structures.

Related papers

• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning [73.18052192964349]
  We develop a theoretical framework that explains how discrete symbolic structures can emerge naturally from continuous neural network training dynamics.<n>By lifting neural parameters to a measure space and modeling training as Wasserstein gradient flow, we show that under geometric constraints, the parameter measure $mu_t$ undergoes two concurrent phenomena.
  arXiv  Detail & Related papers  (2025-06-26T22:40:30Z)
• Geometric Origins of Bias in Deep Neural Networks: A Human Visual System Perspective [1.7315645623674356]
  Bias formation in deep neural networks (DNNs) remains a critical yet poorly understood challenge.<n>Inspired by the human visual system, we propose a geometric analysis framework linking the geometric complexity of class-specific perceptual Manifolds to model bias.<n>To support this analysis, we present the Perceptual-Manifold-Geometry library, designed for calculating the geometric properties of perceptual Manifolds.
  arXiv  Detail & Related papers  (2025-02-17T13:54:02Z)
• Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry [6.369393363312528]
  We promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry.<n>We outline a dictionary between algebro-geometric invariants of varieties, such as dimension, degree, and singularities.<n>Work lays the foundations of a research direction bridging algebraic geometry and deep learning.
  arXiv  Detail & Related papers  (2025-01-31T06:33:58Z)
• Tropical Expressivity of Neural Networks [0.0]
  We use tropical geometry to characterize and study various architectural aspects of neural networks. We present a new algorithm that computes the exact number of their linear regions.
  arXiv  Detail & Related papers  (2024-05-30T15:45:03Z)
• Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning [23.501824517684465]
  This paper provides a framework for understanding why hypercomplex deep learning methods are so successful and how their potential can be exploited. We show that it is possible to derive specific inductive biases in the hypercomplex domains. These biases prove effective in managing the distinctive properties of these domains, as well as the complex structures of multidimensional and multimodal signals.
  arXiv  Detail & Related papers  (2024-05-11T14:41:48Z)
• Fundamental Components of Deep Learning: A category-theoretic approach [0.0]
  This thesis develops a novel mathematical foundation for deep learning based on the language of category theory. We also systematise many existing approaches, placing many existing constructions and concepts under the same umbrella.
  arXiv  Detail & Related papers  (2024-03-13T01:29:40Z)
• Simplicial Representation Learning with Neural $k$-Forms [14.566552361705499]
  This paper focuses on leveraging geometric information from simplicial complexes embedded in $mathbbRn$ using node coordinates. We use differential k-forms in mathbbRn to create representations of simplices, offering interpretability and geometric consistency without message passing. Our method is efficient, versatile, and applicable to various input complexes, including graphs, simplicial complexes, and cell complexes.
  arXiv  Detail & Related papers  (2023-12-13T21:03:39Z)
• Riemannian Residual Neural Networks [58.925132597945634]
  We show how to extend the residual neural network (ResNet) ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks.
  arXiv  Detail & Related papers  (2023-10-16T02:12:32Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• Learning from Protein Structure with Geometric Vector Perceptrons [6.5360079597553025]
  We introduce geometric vector perceptrons, which extend standard dense layers to operate on collections of Euclidean vectors. We demonstrate our approach on two important problems in learning from protein structure: model quality assessment and computational protein design.
  arXiv  Detail & Related papers  (2020-09-03T01:54:25Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Geometrically Principled Connections in Graph Neural Networks [66.51286736506658]
  We argue geometry should remain the primary driving force behind innovation in the emerging field of geometric deep learning. We relate graph neural networks to widely successful computer graphics and data approximation models: radial basis functions (RBFs) We introduce affine skip connections, a novel building block formed by combining a fully connected layer with any graph convolution operator.
  arXiv  Detail & Related papers  (2020-04-06T13:25:46Z)
• Stochastic Flows and Geometric Optimization on the Orthogonal Group [52.50121190744979]
  We present a new class of geometrically-driven optimization algorithms on the orthogonal group $O(d)$. We show that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, flows and metric learning.
  arXiv  Detail & Related papers  (2020-03-30T15:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.