Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis

• URL: http://arxiv.org/abs/2203.01631v1
• Date: Thu, 3 Mar 2022 10:45:53 GMT
• Title: Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis
• Authors: Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
• Abstract summary: We present a fully-connected network and its associated ridgelet transform on the noncompact symmetric space. The ridgelet transform is an analysis operator of a depth-2 continuous network spanned by neurons. Thanks to the coordinate-free reformulation, the role of nonlinear activation functions is revealed to be a wavelet function.
• Score: 10.05944106581306
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric deep learning. Based on the well-established framework of the Helgason-Fourier transform on the noncompact symmetric space, we present a fully-connected network and its associated ridgelet transform on the noncompact symmetric space, covering the hyperbolic neural network (HNN) and the SPDNet as special cases. The ridgelet transform is an analysis operator of a depth-2 continuous network spanned by neurons, namely, it maps an arbitrary given function to the weights of a network. Thanks to the coordinate-free reformulation, the role of nonlinear activation functions is revealed to be a wavelet function, and the reconstruction formula directly yields the universality of the proposed networks.

Related papers

• The Inductive Bias of Convolutional Neural Networks: Locality and Weight Sharing Reshape Implicit Regularization [57.37943479039033]
  We study how architectural inductive bias reshapes the implicit regularization induced by the edge-of-stability phenomenon in gradient descent.<n>We show that locality and weight sharing fundamentally change this picture.
  arXiv  Detail & Related papers  (2026-03-05T04:50:51Z)
• Neural Networks on Symmetric Spaces of Noncompact Type [19.41181017140696]
  We propose a novel approach for developing neural networks on hyperbolic spaces.<n>Our approach is validated on challenging benchmarks for image classification, electroencephalogram (EEG) signal classification, image generation, and natural language inference.
  arXiv  Detail & Related papers  (2026-01-03T07:26:39Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• Symmetry-preserving neural networks in lattice field theories [0.0]
  This thesis deals with neural networks that respect symmetries and presents the advantages in applying them to lattice field theory problems.<n>The benefits of choosing equivariant networks are first illustrated for translational symmetry on a complex scalar field toy model.<n>The discussion is then extended to gauge theories, for which Lattice Gauge Equivariant Convolutional Neural Networks (L-CNNs) are specifically designed ad hoc.
  arXiv  Detail & Related papers  (2025-06-14T13:12:25Z)
• Equivariant non-linear maps for neural networks on homogeneous spaces [8.944149301388551]
  We present a novel framework for non-linear equivariant neural network layers on homogeneous spaces. We derive generalized steerability constraints that any such layer needs to satisfy. We demonstrate how several common equivariant network architectures may be derived from our framework.
  arXiv  Detail & Related papers  (2025-04-29T17:42:56Z)
• Symmetry Adapted Residual Neural Network Diabatization: Conical Intersections in Aniline Photodissociation [1.2365038403958204]
  We present a symmetry adapted neural network (SAResNet) diabatization method to construct quasi-diabatic Hamiltonians. Our SAResNet is applied to construct the full 36-dimensional coupled diabatic potential energy surfaces for aniline N-H bond photodissociation.
  arXiv  Detail & Related papers  (2024-11-03T21:56:25Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks [14.45619075342763]
  We present a systematic method to induce a generalized neural network and its right inverse operator, called the ridgelet transform. Since the ridgelet transform is an inverse, it can describe the arrangement of parameters for the network to represent a target function. We present a new simple proof of the universality by using Schur's lemma in a unified manner covering a wide class of networks.
  arXiv  Detail & Related papers  (2023-10-05T13:30:37Z)
• Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells [52.425628028229156]
  We study the properties of grid cells using recurrent network models. We focus on a simple non-linear recurrent model that underlies the continuous attractor neural networks of grid cells.
  arXiv  Detail & Related papers  (2022-10-06T05:26:49Z)
• Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design [8.250374560598493]
  Hyperbolic neural networks have been popular in the recent past due to their ability to represent hierarchical data sets effectively and efficiently. The challenge in developing these networks lies in the nonlinearity of the embedding space namely, the Hyperbolic space. We present a novel fully hyperbolic neural network which uses the concept of projections (embeddings) followed by an intrinsic aggregation and a nonlinearity all within the hyperbolic space.
  arXiv  Detail & Related papers  (2021-12-03T03:20:27Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Encoding Involutory Invariance in Neural Networks [1.6371837018687636]
  In certain situations, Neural Networks (NN) are trained upon data that obey underlying physical symmetries. In this work, we explore a special kind of symmetry where functions are invariant with respect to involutory linear/affine transformations up to parity. Numerical experiments indicate that the proposed models outperform baseline networks while respecting the imposed symmetry. An adaption of our technique to convolutional NN classification tasks for datasets with inherent horizontal/vertical reflection symmetry has also been proposed.
  arXiv  Detail & Related papers  (2021-06-07T16:07:15Z)
• Fully Hyperbolic Neural Networks [63.22521652077353]
  We propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model. We show that our method has better performance for building both shallow and deep networks.
  arXiv  Detail & Related papers  (2021-05-31T03:36:49Z)
• What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory [19.216784367141972]
  We develop a variational framework to understand the properties of functions learned by deep neural networks with ReLU activation functions fit to data. We derive a representer theorem showing that deep ReLU networks are solutions to regularized data fitting problems in this function space.
  arXiv  Detail & Related papers  (2021-05-07T16:18:22Z)
• From deep to Shallow: Equivalent Forms of Deep Networks in Reproducing Kernel Krein Space and Indefinite Support Vector Machines [63.011641517977644]
  We take a deep network and convert it to an equivalent (indefinite) kernel machine. We then investigate the implications of this transformation for capacity control and uniform convergence. Finally, we analyse the sparsity properties of the flat representation, showing that the flat weights are (effectively) Lp-"norm" regularised with 0p1.
  arXiv  Detail & Related papers  (2020-07-15T03:21:35Z)
• Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms [67.88675386638043]
  The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. We introduce windowed and non-windowed geometric scattering transforms for graphs based upon a very general class of asymmetric wavelets. We show that these asymmetric graph scattering transforms have many of the same theoretical guarantees as their symmetric counterparts.
  arXiv  Detail & Related papers  (2019-11-14T17:23:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.