O$n$ Learning Deep O($n$)-Equivariant Hyperspheres

• URL: http://arxiv.org/abs/2305.15613v7
• Date: Mon, 27 May 2024 16:50:10 GMT
• Title: O$n$ Learning Deep O($n$)-Equivariant Hyperspheres
• Authors: Pavlo Melnyk, Michael Felsberg, Mårten Wadenbäck, Andreas Robinson, Cuong Le,
• Abstract summary: We propose an approach to learning deep features equivariant under the transformations of $n$D reflections and rotations. Namely, we propose O$(n)$-equivariant neurons with spherical decision surfaces that generalize to any dimension $n$. We experimentally verify our theoretical contributions and find that our approach is superior to the competing methods for O$(n)$-equivariant benchmark datasets.
• Score: 18.010317026027028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we utilize hyperspheres and regular $n$-simplexes and propose an approach to learning deep features equivariant under the transformations of $n$D reflections and rotations, encompassed by the powerful group of O$(n)$. Namely, we propose O$(n)$-equivariant neurons with spherical decision surfaces that generalize to any dimension $n$, which we call Deep Equivariant Hyperspheres. We demonstrate how to combine them in a network that directly operates on the basis of the input points and propose an invariant operator based on the relation between two points and a sphere, which as we show, turns out to be a Gram matrix. Using synthetic and real-world data in $n$D, we experimentally verify our theoretical contributions and find that our approach is superior to the competing methods for O$(n)$-equivariant benchmark datasets (classification and regression), demonstrating a favorable speed/performance trade-off. The code is available at https://github.com/pavlo-melnyk/equivariant-hyperspheres.

Related papers

• Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon Transform [4.096453902709292]
  We consider the problem of how efficiently shallow neural networks with the ReLU$k$ activation function can approximate functions from Sobolev spaces. We provide a simple proof of nearly optimal approximation rates in a variety of cases, including when $qleq p$, $pgeq 2$, and $s leq k + (d+1)/2$.
  arXiv  Detail & Related papers  (2024-08-20T16:43:45Z)
• Fast, Expressive SE$(n)$ Equivariant Networks through Weight-Sharing in Position-Orientation Space [15.495593104596399]
  We formalize the notion of weight sharing in convolutional networks as the sharing of message functions over point-pairs. We develop an efficient equivariant group convolutional network for processing 3D point clouds.
  arXiv  Detail & Related papers  (2023-10-04T17:06:32Z)
• Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories [70.90012822736988]
  Existing theories on deep nonparametric regression have shown that when the input data lie on a low-dimensional manifold, deep neural networks can adapt to intrinsic data structures. This paper introduces a relaxed assumption that input data are concentrated around a subset of $mathbbRd$ denoted by $mathcalS$, and the intrinsic dimension $mathcalS$ can be characterized by a new complexity notation -- effective Minkowski dimension.
  arXiv  Detail & Related papers  (2023-06-26T17:13:31Z)
• Equivalence Between SE(3) Equivariant Networks via Steerable Kernels and Group Convolution [90.67482899242093]
  A wide range of techniques have been proposed in recent years for designing neural networks for 3D data that are equivariant under rotation and translation of the input. We provide an in-depth analysis of both methods and their equivalence and relate the two constructions to multiview convolutional networks. We also derive new TFN non-linearities from our equivalence principle and test them on practical benchmark datasets.
  arXiv  Detail & Related papers  (2022-11-29T03:42:11Z)
• Scalable and Equivariant Spherical CNNs by Discrete-Continuous (DISCO) Convolutions [5.8808473430456525]
  No existing spherical convolutional neural network (CNN) framework is both computationally scalable and rotationally equivariant. We develop a hybrid discrete-continuous (DISCO) group convolution that is simultaneously equivariant and computationally to high-resolution. For 4k spherical images we realize a saving of $109$ in computational cost and $104$ in memory usage when compared to the most efficient alternative equivariant spherical convolution.
  arXiv  Detail & Related papers  (2022-09-27T18:00:01Z)
• Lessons from $O(N)$ models in one dimension [0.0]
  Various topics related to the $O(N)$ model in one spacetime dimension (i.e. ordinary quantum mechanics) are considered. The focus is on a pedagogical presentation of quantum field theory methods in a simpler context.
  arXiv  Detail & Related papers  (2021-09-14T11:36:30Z)
• Quantum double aspects of surface code models [77.34726150561087]
  We revisit the Kitaev model for fault tolerant quantum computing on a square lattice with underlying quantum double $D(G)$ symmetry. We show how our constructions generalise to $D(H)$ models based on a finite-dimensional Hopf algebra $H$.
  arXiv  Detail & Related papers  (2021-06-25T17:03:38Z)
• Geometric Deep Learning and Equivariant Neural Networks [0.9381376621526817]
  We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks. We develop gauge equivariant convolutional neural networks on arbitrary manifold $mathcalM$ using principal bundles with structure group $K$ and equivariant maps between sections of associated vector bundles. We analyze several applications of this formalism, including semantic segmentation and object detection networks.
  arXiv  Detail & Related papers  (2021-05-28T15:41:52Z)
• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups [115.58550697886987]
  We provide a completely general algorithm for solving for the equivariant layers of matrix groups. In addition to recovering solutions from other works as special cases, we construct multilayer perceptrons equivariant to multiple groups that have never been tackled before. Our approach outperforms non-equivariant baselines, with applications to particle physics and dynamical systems.
  arXiv  Detail & Related papers  (2021-04-19T17:21:54Z)
• A deep network construction that adapts to intrinsic dimensionality beyond the domain [79.23797234241471]
  We study the approximation of two-layer compositions $f(x) = g(phi(x))$ via deep networks with ReLU activation. We focus on two intuitive and practically relevant choices for $phi$: the projection onto a low-dimensional embedded submanifold and a distance to a collection of low-dimensional sets.
  arXiv  Detail & Related papers  (2020-08-06T09:50:29Z)
• Stochastic Flows and Geometric Optimization on the Orthogonal Group [52.50121190744979]
  We present a new class of geometrically-driven optimization algorithms on the orthogonal group $O(d)$. We show that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, flows and metric learning.
  arXiv  Detail & Related papers  (2020-03-30T15:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.