Riemannian Multinomial Logistics Regression for SPD Neural Networks
- URL: http://arxiv.org/abs/2305.11288v2
- Date: Wed, 20 Mar 2024 15:10:09 GMT
- Title: Riemannian Multinomial Logistics Regression for SPD Neural Networks
- Authors: Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, Xiaojun Wu, Nicu Sebe,
- Abstract summary: We propose a new type of deep neural network for Symmetric Positive Definite (SPD) matrices.
Our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks.
The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification.
- Score: 60.11063972538648
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Deep neural networks for learning Symmetric Positive Definite (SPD) matrices are gaining increasing attention in machine learning. Despite the significant progress, most existing SPD networks use traditional Euclidean classifiers on an approximated space rather than intrinsic classifiers that accurately capture the geometry of SPD manifolds. Inspired by Hyperbolic Neural Networks (HNNs), we propose Riemannian Multinomial Logistics Regression (RMLR) for the classification layers in SPD networks. We introduce a unified framework for building Riemannian classifiers under the metrics pulled back from the Euclidean space, and showcase our framework under the parameterized Log-Euclidean Metric (LEM) and Log-Cholesky Metric (LCM). Besides, our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks. The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification. The code is available at https://github.com/GitZH-Chen/SPDMLR.git.
Related papers
- Matrix Manifold Neural Networks++ [18.385670036798707]
We design fully-connected layers for SPD neural networks.
We propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective.
arXiv Detail & Related papers (2024-05-29T15:47:35Z) - Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs)
Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators.
Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
arXiv Detail & Related papers (2024-05-22T17:23:15Z) - A Lie Group Approach to Riemannian Batch Normalization [59.48083303101632]
This paper establishes a unified framework for normalization techniques on Lie groups.
We focus on Symmetric Positive Definite (SPD), which possess three distinct types of Lie group structures.
Specific normalization layers induced by these Lie groups are then proposed for SPD neural networks.
arXiv Detail & Related papers (2024-03-17T16:24:07Z) - Riemannian Self-Attention Mechanism for SPD Networks [34.794770395408335]
An SPD manifold self-attention mechanism (SMSA) is proposed in this paper.
An SMSA-based geometric learning module (SMSA-GL) is designed for the sake of improving the discrimination of structured representations.
arXiv Detail & Related papers (2023-11-28T12:34:46Z) - Adaptive Riemannian Metrics on SPD Manifolds [67.48576298756996]
Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity of encoding underlying structural correlation in data.
Existing fixed metric tensors might lead to sub-optimal performance for SPD matrices learning, especially for SPD neural networks.
arXiv Detail & Related papers (2023-03-26T18:31:52Z) - A Unified Algebraic Perspective on Lipschitz Neural Networks [88.14073994459586]
This paper introduces a novel perspective unifying various types of 1-Lipschitz neural networks.
We show that many existing techniques can be derived and generalized via finding analytical solutions of a common semidefinite programming (SDP) condition.
Our approach, called SDP-based Lipschitz Layers (SLL), allows us to design non-trivial yet efficient generalization of convex potential layers.
arXiv Detail & Related papers (2023-03-06T14:31:09Z) - DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for
Visual Classification [36.848148506610364]
We propose a new architecture for SPD matrix learning.
To enrich the deep representations, we adopt SPDNet as the backbone.
We then insert several residual-like blocks with shortcut connections to augment the representational capacity of SRAE.
arXiv Detail & Related papers (2022-06-16T07:15:20Z) - Riemannian Local Mechanism for SPD Neural Networks [43.789561494266316]
We argue that it is of utmost importance to ensure the preservation of local geometric information in SPD networks.
We first analyse the convolution operator commonly used for capturing local information in Euclidean deep networks.
Based on this analysis, we define the local information in the SPD manifold and design a multi-scale submanifold block for mining local geometry.
arXiv Detail & Related papers (2022-01-25T07:39:25Z) - Neural Architecture Search of SPD Manifold Networks [79.45110063435617]
We propose a new neural architecture search (NAS) problem of Symmetric Positive Definite (SPD) manifold networks.
We first introduce a geometrically rich and diverse SPD neural architecture search space for an efficient SPD cell design.
We exploit a differentiable NAS algorithm on our relaxed continuous search space for SPD neural architecture search.
arXiv Detail & Related papers (2020-10-27T18:08:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.