Transformation Categorization Based on Group Decomposition Theory Using Parameter Division

• URL: http://arxiv.org/abs/2605.04056v1
• Date: Fri, 10 Apr 2026 03:35:37 GMT
• Title: Transformation Categorization Based on Group Decomposition Theory Using Parameter Division
• Abstract summary: Representation learning seeks meaningful sensory representations without supervision and can model aspects of human development.<n>We study unsupervised categorization of transformations between pairs of inputs under algebraic constraints.<n>We evaluate on image pairs involving rotation, translation, and scale; ablations show that group-decomposition constraints drive appropriate categorization.
• Score: 4.231181719953953
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Representation learning seeks meaningful sensory representations without supervision and can model aspects of human development. Although many neural networks empirically learn useful features, a principled account of what makes a representation "good" remains elusive. We study unsupervised categorization of transformations between pairs of inputs under algebraic constraints. Classical disentanglement favors mutually independent factors and fails when factors are coupled. Our prior Galois-theoretic approach decomposes a group via normal subgroups by learning a product of two transformations with one factor constrained to a normal subgroup, covering both commutative and non-commutative cases. That method, however, relied on auxiliary assumptions (e.g., motion and isometry restrictions) not required by decomposition theory, and ablations did not separate theory-based from auxiliary effects. We propose parameter division for a single transformation: we split its parameter into components, impose homomorphism constraints mapping the full transformation to one component, and identify the normal subgroup as the set of transformations when that component is fixed to the identity. This formulation drops the previous auxiliary assumptions and applies more broadly. We evaluate on image pairs involving rotation, translation, and scale; ablations show that group-decomposition constraints drive appropriate categorization.

Related papers

• Transformers learn factored representations [62.86679034549244]
  Transformers pretrained via next token prediction learn to factor their world into parts.<n>We formalize two representational hypotheses: (1) a representation in the product space of all factors, whose dimension grows exponentially with the number of parts.<n>We derive precise predictions about the geometric structure of activations for each, including the number of subspaces, their dimensionality, and the arrangement of context embeddings within them.<n>This provides a principled explanation for why transformers decompose the world into parts, and suggests that interpretable low dimensional structure may persist even in models trained on complex data.
  arXiv  Detail & Related papers  (2026-02-02T17:49:06Z)
• Unified Structural Embedding of Orbifold Sigma Models [0.0]
  This study introduces a new unified structural framework for orbifold sigma models.<n>The formalism is shown to recover conventional sigma model results in the smooth limit where $G$ approaches the trivial group.<n>Examples demonstrate the applicability, including explicit calculations for the $mathbbC/mathbbZ$ orbifold.
  arXiv  Detail & Related papers  (2025-05-09T20:20:08Z)
• Learning Conditionally Independent Transformations using Normal Subgroups in Group Theory [3.1936317340169817]
  A fundamental challenge in unsupervised representation learning is to separate different transformations in learned feature representations.<n>Existing methods decompose transformations based on algebraic independence, but these methods primarily focus on commutative transformations.<n>We propose a novel approach that leverages normal subgroups to enable the separation of conditionally independent transformations.
  arXiv  Detail & Related papers  (2025-04-06T13:45:43Z)
• Relative Representations: Topological and Geometric Perspectives [50.85040046976025]
  Relative representations are an established approach to zero-shot model stitching.<n>We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations.<n>Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• Lie Group Decompositions for Equivariant Neural Networks [12.139222986297261]
  We show how convolution kernels can be parametrized to build models equivariant with respect to affine transformations. We evaluate the robustness and out-of-distribution generalisation capability of our model on the benchmark affine-invariant classification task.
  arXiv  Detail & Related papers  (2023-10-17T16:04:33Z)
• HiPerformer: Hierarchically Permutation-Equivariant Transformer for Time Series Forecasting [56.95572957863576]
  We propose a hierarchically permutation-equivariant model that considers both the relationship among components in the same group and the relationship among groups. The experiments conducted on real-world data demonstrate that the proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-14T05:11:52Z)
• Equivariant Disentangled Transformation for Domain Generalization under Combination Shift [91.38796390449504]
  Combinations of domains and labels are not observed during training but appear in the test environment. We provide a unique formulation of the combination shift problem based on the concepts of homomorphism, equivariance, and a refined definition of disentanglement.
  arXiv  Detail & Related papers  (2022-08-03T12:31:31Z)
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views? [21.06669693699965]
  We find that the fraction of separable dichotomies is determined by the dimension of the space that is fixed by the group action. We show how this relation extends to operations such as convolutions, element-wise nonlinearities, and global and local pooling.
  arXiv  Detail & Related papers  (2021-10-14T15:46:53Z)
• Commutative Lie Group VAE for Disentanglement Learning [96.32813624341833]
  We view disentanglement learning as discovering an underlying structure that equivariantly reflects the factorized variations shown in data. A simple model named Commutative Lie Group VAE is introduced to realize the group-based disentanglement learning. Experiments show that our model can effectively learn disentangled representations without supervision, and can achieve state-of-the-art performance without extra constraints.
  arXiv  Detail & Related papers  (2021-06-07T07:03:14Z)
• GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement [91.9003001845855]
  VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. We address VAE-based unsupervised disentanglement by leveraging the constraints derived from the Group Theory based definition as the non-probabilistic inductive bias. We train 1800 models covering the most prominent VAE-based models on five datasets to verify the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-02-20T09:49:51Z)
• Learning Irreducible Representations of Noncommutative Lie Groups [3.1727619150610837]
  Recent work has constructed neural networks that are equivariant to continuous symmetry groups such as 2D and 3D rotations. We present two contributions motivated by frontier applications of equivariance beyond rotations and translations.
  arXiv  Detail & Related papers  (2020-06-01T05:14:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.