Identifiable Equivariant Networks are Layerwise Equivariant

• URL: http://arxiv.org/abs/2601.21645v1
• Date: Thu, 29 Jan 2026 12:47:51 GMT
• Title: Identifiable Equivariant Networks are Layerwise Equivariant
• Authors: Vahid Shahverdi, Giovanni Luca Marchetti, Georg Bökman, Kathlén Kohn,
• Abstract summary: We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks.<n>Our results provide a mathematical explanation for the emergence of equivariant structures in the weights of neural networks during training.
• Score: 12.83273311392079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end function is equivariant with respect to group actions on the input and output spaces, there is a parameter choice yielding the same end-to-end function such that its layers are equivariant with respect to some group actions on the latent spaces. Our result assumes that the parameters of the model are identifiable in an appropriate sense. This identifiability property has been established in the literature for a large class of networks, to which our results apply immediately, while it is conjectural for others. The theory we develop is grounded in an abstract formalism, and is therefore architecture-agnostic. Overall, our results provide a mathematical explanation for the emergence of equivariant structures in the weights of neural networks during training -- a phenomenon that is consistently observed in practice.

Related papers

• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• A Characterization Theorem for Equivariant Networks with Point-wise Activations [13.00676132572457]
  We prove that rotation-equivariant networks can only be invariant, as it happens for any network which is equivariant with respect to connected compact groups. We show that feature spaces of disentangled steerable convolutional neural networks are trivial representations.
  arXiv  Detail & Related papers  (2024-01-17T14:30:46Z)
• Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory [64.06519549649495]
  We provide the definition of what we call functionally equivalent features. These features produce equivalent output under certain transformations. We propose an efficient algorithm named Iterative Feature Merging.
  arXiv  Detail & Related papers  (2023-10-10T16:27:12Z)
• Optimization Dynamics of Equivariant and Augmented Neural Networks [2.7918308693131135]
  We investigate the optimization of neural networks on symmetric data. We compare the strategy of constraining the architecture to be equivariant to that of using data augmentation. Our analysis reveals that even in the latter situation, stationary points may be unstable for augmented training although they are stable for the manifestly equivariant models.
  arXiv  Detail & Related papers  (2023-03-23T17:26:12Z)
• Self-Supervised Learning for Group Equivariant Neural Networks [75.62232699377877]
  Group equivariant neural networks are the models whose structure is restricted to commute with the transformations on the input. We propose two concepts for self-supervised tasks: equivariant pretext labels and invariant contrastive loss. Experiments on standard image recognition benchmarks demonstrate that the equivariant neural networks exploit the proposed self-supervised tasks.
  arXiv  Detail & Related papers  (2023-03-08T08:11:26Z)
• Equivariant Transduction through Invariant Alignment [71.45263447328374]
  We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
  arXiv  Detail & Related papers  (2022-09-22T11:19:45Z)
• Bispectral Neural Networks [1.0323063834827415]
  We present a neural network architecture, Bispectral Neural Networks (BNNs) BNNs are able to simultaneously learn groups, their irreducible representations, and corresponding equivariant and complete-invariant maps.
  arXiv  Detail & Related papers  (2022-09-07T18:34:48Z)
• Equivariance and generalization in neural networks [0.0]
  We focus on the consequences of incorporating translational equivariance among the network properties. The benefits of equivariant networks are exemplified by studying a complex scalar field theory. In most of the tasks our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-12-23T12:38:32Z)
• Generalization capabilities of neural networks in lattice applications [0.0]
  We investigate the advantages of adopting translationally equivariant neural networks in favor of non-equivariant ones. We show that our best equivariant architectures can perform and generalize significantly better than their non-equivariant counterparts.
  arXiv  Detail & Related papers  (2021-12-23T11:48:06Z)
• Topographic VAEs learn Equivariant Capsules [84.33745072274942]
  We introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. We demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.
  arXiv  Detail & Related papers  (2021-09-03T09:25:57Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.