Matrix Manifold Neural Networks++

• URL: http://arxiv.org/abs/2405.19206v1
• Date: Wed, 29 May 2024 15:47:35 GMT
• Title: Matrix Manifold Neural Networks++
• Authors: Xuan Son Nguyen, Shuo Yang, Aymeric Histace,
• Abstract summary: We design fully-connected layers for SPD neural networks. We propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective.
• Score: 18.385670036798707
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks (DNNs) on Riemannian manifolds have garnered increasing interest in various applied areas. For instance, DNNs on spherical and hyperbolic manifolds have been designed to solve a wide range of computer vision and nature language processing tasks. One of the key factors that contribute to the success of these networks is that spherical and hyperbolic manifolds have the rich algebraic structures of gyrogroups and gyrovector spaces. This enables principled and effective generalizations of the most successful DNNs to these manifolds. Recently, some works have shown that many concepts in the theory of gyrogroups and gyrovector spaces can also be generalized to matrix manifolds such as Symmetric Positive Definite (SPD) and Grassmann manifolds. As a result, some building blocks for SPD and Grassmann neural networks, e.g., isometric models and multinomial logistic regression (MLR) can be derived in a way that is fully analogous to their spherical and hyperbolic counterparts. Building upon these works, we design fully-connected (FC) and convolutional layers for SPD neural networks. We also develop MLR on Symmetric Positive Semi-definite (SPSD) manifolds, and propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective. We demonstrate the effectiveness of the proposed approach in the human action recognition and node classification tasks.

Related papers

• A Theoretical Study of Neural Network Expressive Power via Manifold Topology [9.054396245059555]
  A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold. In this study, we investigate network expressive power in terms of the latent data manifold. We present a size upper bound of ReLU neural networks.
  arXiv  Detail & Related papers  (2024-10-21T22:10:24Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Deep Extrinsic Manifold Representation for Vision Tasks [8.258646137095395]
  We introduce the trick named Deep Extrinsic Manifold Representation (DEMR) for visual tasks. DEMR incorporates extrinsic manifold embedding into deep neural networks, which helps generate manifold representations. We show that DEMR effectively adapts to point cloud alignment, producing outputs in $ SE(3) $, as well as in illumination subspace learning with outputs on the Grassmann manifold.
  arXiv  Detail & Related papers  (2024-03-31T03:16:08Z)
• Riemannian Residual Neural Networks [58.925132597945634]
  We show how to extend the residual neural network (ResNet) ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks.
  arXiv  Detail & Related papers  (2023-10-16T02:12:32Z)
• Riemannian Multinomial Logistics Regression for SPD Neural Networks [60.11063972538648]
  We propose a new type of deep neural network for Symmetric Positive Definite (SPD) matrices. Our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks. The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification.
  arXiv  Detail & Related papers  (2023-05-18T20:12:22Z)
• Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach [8.003578990152945]
  We propose new models and layers for building neural networks on SPD and Grassmann manifold. We show the effectiveness of our approach in two applications, i.e., human action recognition and knowledge graph completion.
  arXiv  Detail & Related papers  (2023-05-08T09:10:11Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Convolutional Neural Networks on Manifolds: From Graphs and Back [122.06927400759021]
  We propose a manifold neural network (MNN) composed of a bank of manifold convolutional filters and point-wise nonlinearities. To sum up, we focus on the manifold model as the limit of large graphs and construct MNNs, while we can still bring back graph neural networks by the discretization of MNNs.
  arXiv  Detail & Related papers  (2022-10-01T21:17:39Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.