Diffeomorphism-Equivariant Neural Networks

• URL: http://arxiv.org/abs/2602.06695v1
• Date: Fri, 06 Feb 2026 13:31:56 GMT
• Title: Diffeomorphism-Equivariant Neural Networks
• Authors: Josephine Elisabeth Oettinger, Zakhar Shumaylov, Johannes Bostelmann, Jan Lellmann, Carola-Bibiane Schönlieb,
• Abstract summary: We propose a strategy designed to induce diffeomorphism equivariance in pre-trained neural networks via energy-based canonicalisation.<n> Empirical results on segmentation and classification tasks confirm that our approach approximates equivariance and generalises to unseen transformations without relying on extensive data augmentation or retraining.
• Score: 20.100205038541695
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Incorporating group symmetries via equivariance into neural networks has emerged as a robust approach for overcoming the efficiency and data demands of modern deep learning. While most existing approaches, such as group convolutions and averaging-based methods, focus on compact, finite, or low-dimensional groups with linear actions, this work explores how equivariance can be extended to infinite-dimensional groups. We propose a strategy designed to induce diffeomorphism equivariance in pre-trained neural networks via energy-based canonicalisation. Formulating equivariance as an optimisation problem allows us to access the rich toolbox of already established differentiable image registration methods. Empirical results on segmentation and classification tasks confirm that our approach achieves approximate equivariance and generalises to unseen transformations without relying on extensive data augmentation or retraining.

Related papers

• Approximate equivariance via projection-based regularisation [1.5755923640031846]
  Non-equivariant models have regained attention, due to their better runtime performance and imperfect symmetries.<n>This has motivated the development of approximately equivariant models that strike a middle ground between respecting symmetries and fitting the data distribution.<n>We present a mathematical framework for computing the non-equivariance penalty exactly and efficiently in both the spatial and spectral domain.
  arXiv  Detail & Related papers  (2026-01-08T15:35:42Z)
• Parameter-free approximate equivariance for tasks with finite group symmetry [15.964726158869777]
  Equivariant neural networks incorporate symmetries through group actions, embedding them as an inductive bias to improve performance on a wide variety of tasks.<n>We propose a simple zero- parameter approach that imposes approximate equivariance for a finite group in the latent representation, as an additional term in the loss function.<n>We benchmark our approach on three datasets and compare it against several existing equivariant methods, showing that it achieves similar or better performance for a fraction of the parameters.
  arXiv  Detail & Related papers  (2025-06-09T21:23:26Z)
• Improving Equivariant Model Training via Constraint Relaxation [31.507956579770088]
  We propose a novel framework for improving the optimization of such models by relaxing the hard equivariance constraint during training.<n>We provide experimental results on different state-of-the-art network architectures, demonstrating how this training framework can result in equivariant models with improved generalization performance.
  arXiv  Detail & Related papers  (2024-08-23T17:35:08Z)
• Affine-Transformation-Invariant Image Classification by Differentiable Arithmetic Distribution Module [8.125023712173686]
  Convolutional Neural Networks (CNNs) have achieved promising results in image classification. CNNs are vulnerable to affine transformations including rotation, translation, flip and shuffle. In this work, we introduce a more robust substitute by incorporating distribution learning techniques.
  arXiv  Detail & Related papers  (2023-09-01T22:31:32Z)
• On the Implicit Bias of Linear Equivariant Steerable Networks [9.539074889921935]
  We study the implicit bias of gradient flow on linear equivariant steerable networks in group-invariant binary classification. Under a unitary assumption on the input representation, we establish the equivalence between steerable networks and data augmentation.
  arXiv  Detail & Related papers  (2023-03-07T19:37:35Z)
• Architectural Optimization over Subgroups for Equivariant Neural Networks [0.0]
  We propose equivariance relaxation morphism and $[G]$-mixed equivariant layer to operate with equivariance constraints on a subgroup. We present evolutionary and differentiable neural architecture search (NAS) algorithms that utilize these mechanisms respectively for equivariance-aware architectural optimization.
  arXiv  Detail & Related papers  (2022-10-11T14:37:29Z)
• Topographic VAEs learn Equivariant Capsules [84.33745072274942]
  We introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. We demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.
  arXiv  Detail & Related papers  (2021-09-03T09:25:57Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)
• Invariant Deep Compressible Covariance Pooling for Aerial Scene Categorization [80.55951673479237]
  We propose a novel invariant deep compressible covariance pooling (IDCCP) to solve nuisance variations in aerial scene categorization. We conduct extensive experiments on the publicly released aerial scene image data sets and demonstrate the superiority of this method compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-11T11:13:07Z)
• Learning Invariances in Neural Networks [51.20867785006147]
  We show how to parameterize a distribution over augmentations and optimize the training loss simultaneously with respect to the network parameters and augmentation parameters. We can recover the correct set and extent of invariances on image classification, regression, segmentation, and molecular property prediction from a large space of augmentations.
  arXiv  Detail & Related papers  (2020-10-22T17:18:48Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data [52.78581260260455]
  We propose a general method to construct a convolutional layer that is equivariant to transformations from any specified Lie group. We apply the same model architecture to images, ball-and-stick molecular data, and Hamiltonian dynamical systems.
  arXiv  Detail & Related papers  (2020-02-25T17:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.