Learning Conditionally Independent Transformations using Normal Subgroups in Group Theory

• URL: http://arxiv.org/abs/2504.04490v1
• Date: Sun, 06 Apr 2025 13:45:43 GMT
• Title: Learning Conditionally Independent Transformations using Normal Subgroups in Group Theory
• Abstract summary: A fundamental challenge in unsupervised representation learning is to separate different transformations in learned feature representations.<n>Existing methods decompose transformations based on algebraic independence, but these methods primarily focus on commutative transformations.<n>We propose a novel approach that leverages normal subgroups to enable the separation of conditionally independent transformations.
• Score: 3.1936317340169817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans develop certain cognitive abilities to recognize objects and their transformations without explicit supervision, highlighting the importance of unsupervised representation learning. A fundamental challenge in unsupervised representation learning is to separate different transformations in learned feature representations. Although algebraic approaches have been explored, a comprehensive theoretical framework remains underdeveloped. Existing methods decompose transformations based on algebraic independence, but these methods primarily focus on commutative transformations and do not extend to cases where transformations are conditionally independent but noncommutative. To extend current representation learning frameworks, we draw inspiration from Galois theory, where the decomposition of groups through normal subgroups provides an approach for the analysis of structured transformations. Normal subgroups naturally extend commutativity under certain conditions and offer a foundation for the categorization of transformations, even when they do not commute. In this paper, we propose a novel approach that leverages normal subgroups to enable the separation of conditionally independent transformations, even in the absence of commutativity. Through experiments on geometric transformations in images, we show that our method successfully categorizes conditionally independent transformations, such as rotation and translation, in an unsupervised manner, suggesting a close link between group decomposition via normal subgroups and transformation categorization in representation learning.

Related papers

• Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits [0.0]
  We investigate how small transformers learn modular integer multiplication over composite moduli.<n>We propose a localized generalization of Group Composition via Representation (GCR) that suggests the learned computation does not rely on a single global representation space.
  arXiv  Detail & Related papers  (2026-07-08T06:49:48Z)
• Transformation Categorization Based on Group Decomposition Theory Using Parameter Division [4.231181719953953]
  Representation learning seeks meaningful sensory representations without supervision and can model aspects of human development.<n>We study unsupervised categorization of transformations between pairs of inputs under algebraic constraints.<n>We evaluate on image pairs involving rotation, translation, and scale; ablations show that group-decomposition constraints drive appropriate categorization.
  arXiv  Detail & Related papers  (2026-04-10T03:35:37Z)
• Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions [18.11344454990819]
  We learn equivariant embeddings from observation pairs $(mathbfy, g cdot mathbfy)$, where $g$ is drawn from a finite group acting on the data.<n>Our method jointly learns a latent space and a group representation in which group actions correspond to invertible linear maps.
  arXiv  Detail & Related papers  (2025-10-24T17:59:46Z)
• Relative Representations: Topological and Geometric Perspectives [53.88896255693922]
  Relative representations are an established approach to zero-shot model stitching. We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations. Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• Algebras of actions in an agent's representations of the world [51.06229789727133]
  We use our framework to reproduce the symmetry-based representations from the symmetry-based disentangled representation learning formalism. We then study the algebras of the transformations of worlds with features that occur in simple reinforcement learning scenarios. Using computational methods, that we developed, we extract the algebras of the transformations of these worlds and classify them according to their properties.
  arXiv  Detail & Related papers  (2023-10-02T18:24:51Z)
• Equivariant Disentangled Transformation for Domain Generalization under Combination Shift [91.38796390449504]
  Combinations of domains and labels are not observed during training but appear in the test environment. We provide a unique formulation of the combination shift problem based on the concepts of homomorphism, equivariance, and a refined definition of disentanglement.
  arXiv  Detail & Related papers  (2022-08-03T12:31:31Z)
• Transformation Coding: Simple Objectives for Equivariant Representations [17.544323284367927]
  We present a non-generative approach to deep representation learning that seeks equivariant deep embedding through simple objectives. In contrast to existing equivariant networks, our transformation coding approach does not constrain the choice of the feed-forward layer or the architecture.
  arXiv  Detail & Related papers  (2022-02-19T01:43:13Z)
• Unsupervised Learning of Group Invariant and Equivariant Representations [10.252723257176566]
  We extend group invariant and equivariant representation learning to the field of unsupervised deep learning. We propose a general learning strategy based on an encoder-decoder framework in which the latent representation is separated in an invariant term and an equivariant group action component. The key idea is that the network learns to encode and decode data to and from a group-invariant representation by additionally learning to predict the appropriate group action to align input and output pose to solve the reconstruction task.
  arXiv  Detail & Related papers  (2022-02-15T16:44:21Z)
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views? [21.06669693699965]
  We find that the fraction of separable dichotomies is determined by the dimension of the space that is fixed by the group action. We show how this relation extends to operations such as convolutions, element-wise nonlinearities, and global and local pooling.
  arXiv  Detail & Related papers  (2021-10-14T15:46:53Z)
• Commutative Lie Group VAE for Disentanglement Learning [96.32813624341833]
  We view disentanglement learning as discovering an underlying structure that equivariantly reflects the factorized variations shown in data. A simple model named Commutative Lie Group VAE is introduced to realize the group-based disentanglement learning. Experiments show that our model can effectively learn disentangled representations without supervision, and can achieve state-of-the-art performance without extra constraints.
  arXiv  Detail & Related papers  (2021-06-07T07:03:14Z)
• GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement [91.9003001845855]
  VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. We address VAE-based unsupervised disentanglement by leveraging the constraints derived from the Group Theory based definition as the non-probabilistic inductive bias. We train 1800 models covering the most prominent VAE-based models on five datasets to verify the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-02-20T09:49:51Z)
• LieTransformer: Equivariant self-attention for Lie Groups [49.9625160479096]
  Group equivariant neural networks are used as building blocks of group invariant neural networks. We extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
  arXiv  Detail & Related papers  (2020-12-20T11:02:49Z)
• Learning Irreducible Representations of Noncommutative Lie Groups [3.1727619150610837]
  Recent work has constructed neural networks that are equivariant to continuous symmetry groups such as 2D and 3D rotations. We present two contributions motivated by frontier applications of equivariance beyond rotations and translations.
  arXiv  Detail & Related papers  (2020-06-01T05:14:29Z)
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data [52.78581260260455]
  We propose a general method to construct a convolutional layer that is equivariant to transformations from any specified Lie group. We apply the same model architecture to images, ball-and-stick molecular data, and Hamiltonian dynamical systems.
  arXiv  Detail & Related papers  (2020-02-25T17:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.