A Lie Group Approach to Riemannian Batch Normalization

• URL: http://arxiv.org/abs/2403.11261v1
• Date: Sun, 17 Mar 2024 16:24:07 GMT
• Title: A Lie Group Approach to Riemannian Batch Normalization
• Authors: Ziheng Chen, Yue Song, Yunmei Liu, Nicu Sebe,
• Abstract summary: This paper establishes a unified framework for normalization techniques on Lie groups. We focus on Symmetric Positive Definite (SPD), which possess three distinct types of Lie group structures. Specific normalization layers induced by these Lie groups are then proposed for SPD neural networks.
• Score: 59.48083303101632
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Manifold-valued measurements exist in numerous applications within computer vision and machine learning. Recent studies have extended Deep Neural Networks (DNNs) to manifolds, and concomitantly, normalization techniques have also been adapted to several manifolds, referred to as Riemannian normalization. Nonetheless, most of the existing Riemannian normalization methods have been derived in an ad hoc manner and only apply to specific manifolds. This paper establishes a unified framework for Riemannian Batch Normalization (RBN) techniques on Lie groups. Our framework offers the theoretical guarantee of controlling both the Riemannian mean and variance. Empirically, we focus on Symmetric Positive Definite (SPD) manifolds, which possess three distinct types of Lie group structures. Using the deformation concept, we generalize the existing Lie groups on SPD manifolds into three families of parameterized Lie groups. Specific normalization layers induced by these Lie groups are then proposed for SPD neural networks. We demonstrate the effectiveness of our approach through three sets of experiments: radar recognition, human action recognition, and electroencephalography (EEG) classification. The code is available at https://github.com/GitZH-Chen/LieBN.git.

Related papers

• RMLR: Extending Multinomial Logistic Regression into General Geometries [64.16104856124029]
  Our framework only requires minimal geometric properties, thus exhibiting broad applicability. We develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric.
  arXiv  Detail & Related papers  (2024-09-28T18:38:21Z)
• Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
  Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.
  arXiv  Detail & Related papers  (2024-07-15T07:11:44Z)
• Riemannian Multinomial Logistics Regression for SPD Neural Networks [60.11063972538648]
  We propose a new type of deep neural network for Symmetric Positive Definite (SPD) matrices. Our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks. The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification.
  arXiv  Detail & Related papers  (2023-05-18T20:12:22Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations [77.86290991564829]
  Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. We study a particular sequence of maps between manifold, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between implementing neural networks of practical interest.
  arXiv  Detail & Related papers  (2021-12-17T11:43:30Z)
• GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition [2.817412580574242]
  We propose a novel method for representation and classification of two-person interactions from 3D skeleton sequences. We show that the proposed method achieves competitive results in two-person interaction recognition on three benchmarks for 3D human activity understanding.
  arXiv  Detail & Related papers  (2021-11-25T13:57:43Z)
• ManifoldNorm: Extending normalizations on Riemannian Manifolds [18.073864874996534]
  We propose a general normalization techniques for manifold valued data. We show that our proposed manifold normalization technique have special cases including popular batch norm and group norm techniques.
  arXiv  Detail & Related papers  (2020-03-30T23:45:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.