On Universality Classes of Equivariant Networks

• URL: http://arxiv.org/abs/2506.02293v1
• Date: Mon, 02 Jun 2025 22:07:52 GMT
• Title: On Universality Classes of Equivariant Networks
• Authors: Marco Pacini, Gabriele Santin, Bruno Lepri, Shubhendu Trivedi,
• Abstract summary: We investigate the approximation power of equivariant neural networks beyond separation constraints.<n>We show that separation power does not fully capture expressivity.<n>We identify settings where shallow equivariant networks do achieve universality.
• Score: 9.137637807153464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Equivariant neural networks provide a principled framework for incorporating symmetry into learning architectures and have been extensively analyzed through the lens of their separation power, that is, the ability to distinguish inputs modulo symmetry. This notion plays a central role in settings such as graph learning, where it is often formalized via the Weisfeiler-Leman hierarchy. In contrast, the universality of equivariant models-their capacity to approximate target functions-remains comparatively underexplored. In this work, we investigate the approximation power of equivariant neural networks beyond separation constraints. We show that separation power does not fully capture expressivity: models with identical separation power may differ in their approximation ability. To demonstrate this, we characterize the universality classes of shallow invariant networks, providing a general framework for understanding which functions these architectures can approximate. Since equivariant models reduce to invariant ones under projection, this analysis yields sufficient conditions under which shallow equivariant networks fail to be universal. Conversely, we identify settings where shallow models do achieve separation-constrained universality. These positive results, however, depend critically on structural properties of the symmetry group, such as the existence of adequate normal subgroups, which may not hold in important cases like permutation symmetry.

Related papers

• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• Separation Power of Equivariant Neural Networks [11.906285279109477]
  We analyze the separation power of equivariant neural networks, such as convolutional and permutation-invariant networks.<n>All non-polynomial activations, including ReLU and sigmoid, are equivalent in expressivity and reach maximum separation power.
  arXiv  Detail & Related papers  (2024-06-13T09:52:44Z)
• A Characterization Theorem for Equivariant Networks with Point-wise Activations [13.00676132572457]
  We prove that rotation-equivariant networks can only be invariant, as it happens for any network which is equivariant with respect to connected compact groups. We show that feature spaces of disentangled steerable convolutional neural networks are trivial representations.
  arXiv  Detail & Related papers  (2024-01-17T14:30:46Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Equivariance and generalization in neural networks [0.0]
  We focus on the consequences of incorporating translational equivariance among the network properties. The benefits of equivariant networks are exemplified by studying a complex scalar field theory. In most of the tasks our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-12-23T12:38:32Z)
• Generalization capabilities of neural networks in lattice applications [0.0]
  We investigate the advantages of adopting translationally equivariant neural networks in favor of non-equivariant ones. We show that our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-12-23T11:48:06Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)
• Exact solutions of interacting dissipative systems via weak symmetries [77.34726150561087]
  We analytically diagonalize the Liouvillian of a class Markovian dissipative systems with arbitrary strong interactions or nonlinearity. This enables an exact description of the full dynamics and dissipative spectrum. Our method is applicable to a variety of other systems, and could provide a powerful new tool for the study of complex driven-dissipative quantum systems.
  arXiv  Detail & Related papers  (2021-09-27T17:45:42Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Generalization capabilities of translationally equivariant neural networks [0.0]
  In this work, we focus on complex scalar field theory on a two-dimensional lattice and investigate the benefits of using group equivariant convolutional neural network architectures. For a meaningful comparison, we conduct a systematic search for equivariant and non-equivariant neural network architectures and apply them to various regression and classification tasks. We demonstrate that our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-03-26T18:53:36Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)
• Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators [72.62940905965267]
  Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. Are CF-INNs universal approximators for invertible functions? We prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes.
  arXiv  Detail & Related papers  (2020-06-20T02:07:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.