Complexity Induction: Compositional Generalization via Structured Training Distortion

• URL: http://arxiv.org/abs/2608.21464v2
• Date: Thu, 27 Aug 2026 13:58:48 GMT
• Title: Complexity Induction: Compositional Generalization via Structured Training Distortion
• Abstract summary: We show that structured distortion of training data can induce compositional generalization without architectural modification.<n>Results suggest that structured complication of training signals can influence both the internal organization of learned representations and their compositional interpretation.
• Score: 51.56484100374058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We demonstrate that structured distortion of training data - which we term complexity induction - can induce compositional generalization in a standard CNN classifier without architectural modification. Using synthetic images of colored geometric shapes, we encode classes as flat string labels (e.g., "red-circle") with no explicit attribute decomposition, and exclude certain color-shape combinations from training entirely. We apply two distortion methods derived from Jaccard string similarity between class names: mixed labels (soft target distributions encoding inter-class overlap) and expanded dataset (false training samples with structurally motivated incorrect labels). Both methods induce the ability to predict unseen class combinations, and act at different levels: mixed labels activate the classifier for unseen combinations by exploiting the CNN's natural embedding structure, while expanded training improves the embedding factorization itself. A control with random (unstructured) false labels confirms that the effect depends on the structure of the distortion, not on noise per se. These results suggest that structured complication of training signals can influence both the internal organization of learned representations and their compositional interpretation - a principle that may underlie the role of natural language in cognitive development.

Related papers

• Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads [5.703301837612397]
  A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings.<n>No unified theory guides the basic design decisions.<n>We show that normalization is gauge fixing and that whitening pins the interaction modes up to permutation and sign.
  arXiv  Detail & Related papers  (2026-08-12T05:09:44Z)
• Layerwise Dynamics for In-Context Classification in Transformers [27.052342498221748]
  We study multi-class linear classification in the hard no-margin regime.<n>We make computation identifiable by enforcing feature- and label-permutation equivariance at every layer.<n>This enables interpretability while maintaining functional equivalence.
  arXiv  Detail & Related papers  (2026-04-13T15:20:41Z)
• From Static Structures to Ensembles: Studying and Harnessing Protein Structure Tokenization [15.864659611818661]
  Protein structure tokenization converts 3D structures into discrete or vectorized representations.<n>Despite many recent works on structure tokenization, the properties of the underlying discrete representations are not well understood.<n>We show that the successful utilization of structural tokens in a language model for structure prediction depends on using rich, pre-trained sequence embeddings.
  arXiv  Detail & Related papers  (2025-11-13T07:58:24Z)
• How Compositional Generalization and Creativity Improve as Diffusion Models are Trained [82.08869888944324]
  How many samples do generative models need in order to learn composition rules?<n>What signal in the data is exploited to learn those rules?<n>We discuss connections between the hierarchical clustering mechanism we introduce here and the renormalization group in physics.
  arXiv  Detail & Related papers  (2025-02-17T18:06:33Z)
• Learning Structured Representations with Hyperbolic Embeddings [22.95613852886361]
  We propose HypStructure: a Hyperbolic Structured regularization approach to accurately embed the label hierarchy into the learned representations.<n>Experiments on several large-scale vision benchmarks demonstrate the efficacy of HypStructure in reducing distortion.<n>For a better understanding of structured representation, we perform eigenvalue analysis that links the representation geometry to improved Out-of-Distribution (OOD) detection performance.
  arXiv  Detail & Related papers  (2024-12-02T00:56:44Z)
• When does compositional structure yield compositional generalization? A kernel theory [0.0]
  We present a theory of compositional generalization in kernel models with fixed, compositionally structured representations.<n>We identify novel failure modes in compositional generalization that arise from biases in the training data.<n>This work examines how statistical structure in the training data can affect compositional generalization.
  arXiv  Detail & Related papers  (2024-05-26T00:50:11Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Commutative Lie Group VAE for Disentanglement Learning [96.32813624341833]
  We view disentanglement learning as discovering an underlying structure that equivariantly reflects the factorized variations shown in data. A simple model named Commutative Lie Group VAE is introduced to realize the group-based disentanglement learning. Experiments show that our model can effectively learn disentangled representations without supervision, and can achieve state-of-the-art performance without extra constraints.
  arXiv  Detail & Related papers  (2021-06-07T07:03:14Z)
• Linguistically Driven Graph Capsule Network for Visual Question Reasoning [153.76012414126643]
  We propose a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network" The compositional process is guided by the linguistic parse tree. Specifically, we bind each capsule in the lowest layer to bridge the linguistic embedding of a single word in the original question with visual evidence. Experiments on the CLEVR dataset, CLEVR compositional generation test, and FigureQA dataset demonstrate the effectiveness and composition generalization ability of our end-to-end model.
  arXiv  Detail & Related papers  (2020-03-23T03:34:25Z)
• Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification [139.44681304276]
  zero-shot learning aims to classify unseen categories for which no data is available during training. Generative Adrial Networks synthesize unseen class features by leveraging class-specific semantic embeddings. We propose to enforce semantic consistency at all stages of zero-shot learning: training, feature synthesis and classification.
  arXiv  Detail & Related papers  (2020-03-17T17:34:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.