Universality of group convolutional neural networks based on ridgelet analysis on groups

• URL: http://arxiv.org/abs/2205.14819v1
• Date: Mon, 30 May 2022 02:52:22 GMT
• Title: Universality of group convolutional neural networks based on ridgelet analysis on groups
• Authors: Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
• Abstract summary: We investigate the approximation property of group convolutional neural networks (GCNNs) based on the ridgelet theory. We formulate a versatile GCNN as a nonlinear mapping between group representations.
• Score: 10.05944106581306
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the approximation property of group convolutional neural networks (GCNNs) based on the ridgelet theory. We regard a group convolution as a matrix element of a group representation, and formulate a versatile GCNN as a nonlinear mapping between group representations, which covers typical GCNN literatures such as a cyclic convolution on a multi-channel image, permutation-invariant datasets (Deep Sets), and $\mathrm{E}(n)$-equivariant convolutions. The ridgelet transform is an analysis operator of a depth-2 network, namely, it maps an arbitrary given target function $f$ to the weight $\gamma$ of a network $S[\gamma]$ so that the network represents the function as $S[\gamma]=f$. It has been known only for fully-connected networks, and this study is the first to present the ridgelet transform for (G)CNNs. Since the ridgelet transform is given as a closed-form integral operator, it provides a constructive proof of the $cc$-universality of GCNNs. Unlike previous universality arguments on CNNs, we do not need to convert/modify the networks into other universal approximators such as invariant polynomials and fully-connected networks.

Related papers

• A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks [14.45619075342763]
  The ridgelet transform is a pseudo-inverse operator that maps a given function $f$ to the parameter distribution $gamma$. For depth-2 fully-connected networks on a Euclidean space, the ridgelet transform has been discovered up to the closed-form expression. We derive transforms for a variety of modern networks such as networks on finite fields $mathbbF_p$, group convolutional networks on abstract Hilbert space $mathcalH$, fully-connected networks on noncompact symmetric spaces $G/K$, and pooling layers.
  arXiv  Detail & Related papers  (2024-02-25T04:30:04Z)
• Degree-based stratification of nodes in Graph Neural Networks [66.17149106033126]
  We modify the Graph Neural Network (GNN) architecture so that the weight matrices are learned, separately, for the nodes in each group. This simple-to-implement modification seems to improve performance across datasets and GNN methods.
  arXiv  Detail & Related papers  (2023-12-16T14:09:23Z)
• Non Commutative Convolutional Signal Models in Neural Networks: Stability to Small Deformations [111.27636893711055]
  We study the filtering and stability properties of non commutative convolutional filters. Our results have direct implications for group neural networks, multigraph neural networks and quaternion neural networks.
  arXiv  Detail & Related papers  (2023-10-05T20:27:22Z)
• Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks [14.45619075342763]
  We present a systematic method to induce a generalized neural network and its right inverse operator, called the ridgelet transform. Since the ridgelet transform is an inverse, it can describe the arrangement of parameters for the network to represent a target function. We present a new simple proof of the universality by using Schur's lemma in a unified manner covering a wide class of networks.
  arXiv  Detail & Related papers  (2023-10-05T13:30:37Z)
• A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis [15.376680573592997]
  Modern neural networks are capable of incorporating more than just translation symmetry. We generalize the Strong Lottery Ticket Hypothesis (SLTH) to functions that preserve the action of the group $G$. We prove our theory by overparametrized $textE(2)$-steerable CNNs and message passing GNNs.
  arXiv  Detail & Related papers  (2022-06-09T04:40:18Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Revisiting Transformation Invariant Geometric Deep Learning: Are Initial Representations All You Need? [80.86819657126041]
  We show that transformation-invariant and distance-preserving initial representations are sufficient to achieve transformation invariance. Specifically, we realize transformation-invariant and distance-preserving initial point representations by modifying multi-dimensional scaling. We prove that TinvNN can strictly guarantee transformation invariance, being general and flexible enough to be combined with the existing neural networks.
  arXiv  Detail & Related papers  (2021-12-23T03:52:33Z)
• A New Neural Network Architecture Invariant to the Action of Symmetry Subgroups [11.812645659940237]
  We propose a $G$-invariant neural network that approximates functions invariant to the action of a given permutation subgroup on input data. The key element of the proposed network architecture is a new $G$-invariant transformation module, which produces a $G$-invariant latent representation of the input data.
  arXiv  Detail & Related papers  (2020-12-11T16:19:46Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• A Unified View on Graph Neural Networks as Graph Signal Denoising [49.980783124401555]
  Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data. In this work, we establish mathematically that the aggregation processes in a group of representative GNN models can be regarded as solving a graph denoising problem. We instantiate a novel GNN model, ADA-UGNN, derived from UGNN, to handle graphs with adaptive smoothness across nodes.
  arXiv  Detail & Related papers  (2020-10-05T04:57:18Z)
• A Computationally Efficient Neural Network Invariant to the Action of Symmetry Subgroups [12.654871396334668]
  A new $G$-invariant transformation module produces a $G$-invariant latent representation of the input data. This latent representation is then processed with a multi-layer perceptron in the network. We prove the universality of the proposed architecture, discuss its properties and highlight its computational and memory efficiency.
  arXiv  Detail & Related papers  (2020-02-18T12:50:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.