Stability Analysis of Equivariant Convolutional Representations Through The Lens of Equivariant Multi-layered CKNs

• URL: http://arxiv.org/abs/2408.04277v1
• Date: Thu, 8 Aug 2024 07:31:22 GMT
• Title: Stability Analysis of Equivariant Convolutional Representations Through The Lens of Equivariant Multi-layered CKNs
• Authors: Soutrik Roy Chowdhury,
• Abstract summary: We construct and theoretically analyse group equivariant kernel networks (CKNs) We study the stability analysis of such equiv-CNNs under the action of diffeomorphism. The goal is to analyse the geometry of inductive biases of equiv-CNNs through the lens of reproducing kernel Hilbert spaces (RKHSs)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we construct and theoretically analyse group equivariant convolutional kernel networks (CKNs) which are useful in understanding the geometry of (equivariant) CNNs through the lens of reproducing kernel Hilbert spaces (RKHSs). We then proceed to study the stability analysis of such equiv-CKNs under the action of diffeomorphism and draw a connection with equiv-CNNs, where the goal is to analyse the geometry of inductive biases of equiv-CNNs through the lens of reproducing kernel Hilbert spaces (RKHSs). Traditional deep learning architectures, including CNNs, trained with sophisticated optimization algorithms is vulnerable to perturbations, including `adversarial examples'. Understanding the RKHS norm of such models through CKNs is useful in designing the appropriate architecture and can be useful in designing robust equivariant representation learning models.

Related papers

• Convolutional Filtering with RKHS Algebras [110.06688302593349]
  We develop a theory of convolutional signal processing and neural networks for Reproducing Kernel Hilbert Spaces (RKHS) We show that any RKHS allows the formal definition of multiple algebraic convolutional models. We present a set of numerical experiments on real data in which wireless coverage is predicted from measurements captured by unmaned aerial vehicles.
  arXiv  Detail & Related papers  (2024-11-02T18:53:44Z)
• On the Convergence Analysis of Over-Parameterized Variational Autoencoders: A Neural Tangent Kernel Perspective [7.580900499231056]
  Variational Auto-Encoders (VAEs) have emerged as powerful probabilistic models for generative tasks. This paper provides a mathematical proof of VAE under mild assumptions. We also establish a novel connection between the optimization problem faced by over-Eized SNNs and the Kernel Ridge (KRR) problem.
  arXiv  Detail & Related papers  (2024-09-09T06:10:31Z)
• E(n) Equivariant Topological Neural Networks [10.603892843083173]
  Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently as a promising tool for addressing this issue. This paper introduces E(n)-Equivariant Topological Neural Networks (ETNNs) ETNNs incorporate geometric node features while respecting rotation, reflection, and translation.
  arXiv  Detail & Related papers  (2024-05-24T10:55:38Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• SO(2) and O(2) Equivariance in Image Recognition with Bessel-Convolutional Neural Networks [63.24965775030674]
  This work presents the development of Bessel-convolutional neural networks (B-CNNs) B-CNNs exploit a particular decomposition based on Bessel functions to modify the key operation between images and filters. Study is carried out to assess the performances of B-CNNs compared to other methods.
  arXiv  Detail & Related papers  (2023-04-18T18:06:35Z)
• Geometrical aspects of lattice gauge equivariant convolutional neural networks [0.0]
  Lattice gauge equivariant convolutional neural networks (L-CNNs) are a framework for convolutional neural networks that can be applied to non-Abelian lattice gauge theories.
  arXiv  Detail & Related papers  (2023-03-20T20:49:08Z)
• Theory for Equivariant Quantum Neural Networks [0.0]
  We present a theoretical framework to design equivariant quantum neural networks (EQNNs) for essentially any relevant symmetry group. Our framework can be readily applied to virtually all areas of quantum machine learning.
  arXiv  Detail & Related papers  (2022-10-16T15:42:21Z)
• Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics [6.940097162264939]
  randomness makes SCNs more likely to generate approximate linear correlative nodes that are redundant and low quality. In light of a fundamental principle in machine learning, that is, a model with fewer parameters holds improved generalization. This paper proposes orthogonal SCN, termed OSCN, to filtrate out the low-quality hidden nodes for network structure reduction.
  arXiv  Detail & Related papers  (2022-05-26T07:07:26Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• Lorentz Group Equivariant Neural Network for Particle Physics [58.56031187968692]
  We present a neural network architecture that is fully equivariant with respect to transformations under the Lorentz group. For classification tasks in particle physics, we demonstrate that such an equivariant architecture leads to drastically simpler models that have relatively few learnable parameters.
  arXiv  Detail & Related papers  (2020-06-08T17:54:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.