Compositional Generalization via Structural Identification in a Category-Theoretic Framework

• URL: http://arxiv.org/abs/2608.26465v1
• Date: Wed, 26 Aug 2026 23:40:59 GMT
• Title: Compositional Generalization via Structural Identification in a Category-Theoretic Framework
• Abstract summary: We ask which structural or lexical identifications make held-out COGS examples admissible from the structures observed in training.<n>Across 21 COGS generalization types, admissibility follows distinct identification profiles, while residual failures separate unsupported templates.
• Score: 9.486586593065892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compositional generalization is usually evaluated through model accuracy. We instead ask which structural or lexical identifications make held-out COGS examples admissible from the structures observed in training. Sentences are represented as functors from syntactic addresses to lexical tokens, and selective collapses induce Kan extensions that propagate observed associations. Across 21 COGS generalization types, admissibility follows distinct identification profiles, while residual failures separate unsupported structural templates. These data-side diagnoses characterize what the training corpus licenses under specified identifications, without training a predictive model.

Related papers

• Complexity Induction: Compositional Generalization via Structured Training Distortion [51.56484100374058]
  We show that structured distortion of training data can induce compositional generalization without architectural modification.<n>Results suggest that structured complication of training signals can influence both the internal organization of learned representations and their compositional interpretation.
  arXiv  Detail & Related papers  (2026-08-21T00:45:02Z)
• Annotating Korean adnominal ending constructions in corpus data: Beyond relative-clause identification [6.030221684048552]
  Korean adnominal ending textttETM occurs in diverse noun-modifying constructions.<n>This paper argues that textttETM is not a direct marker of relative-clause structure, but a morphological exponent.
  arXiv  Detail & Related papers  (2026-07-04T03:21:50Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Statistical and structural identifiability in representation learning [31.990445585569688]
  We formalize two distinct concepts: statistical identifiability and structural identifiability.<n>We prove a statistical $$-near-identifiability result for the representations of models with nonlinear decoders.<n>With additional assumptions on the data-generating process, statistical identifiability extends to structural identifiability.
  arXiv  Detail & Related papers  (2026-03-12T14:19:58Z)
• Grammaticality Judgments in Humans and Language Models: Revisiting Generative Grammar with LLMs [0.0]
  In traditional generative grammar, systematic contrasts in grammaticality such as subject-auxiliary inversion and the licensing of parasitic gaps are taken as evidence for an internal, hierarchical grammar.<n>We test whether large language models (LLMs), trained only on surface forms, reproduce these contrasts in ways that imply an underlying structural representation.
  arXiv  Detail & Related papers  (2025-12-11T09:17:35Z)
• Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers [74.96551626420188]
  Transformers trained on natural language data have been shown to learn its hierarchical structure and generalize to sentences with unseen syntactic structures.<n>We investigate sources of inductive bias in transformer models and their training that could cause such generalization behavior to emerge.
  arXiv  Detail & Related papers  (2024-04-25T07:10:29Z)
• SLOG: A Structural Generalization Benchmark for Semantic Parsing [68.19511282584304]
  The goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical generalization, the interpretation of novel lexical items in syntactic structures familiar from training, are often underrepresented. We introduce SLOG, a semantic parsing dataset that extends COGS with 17 structural generalization cases.
  arXiv  Detail & Related papers  (2023-10-23T15:39:09Z)
• Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task [70.29624135819884]
  We study the extent to which BERT is able to perform lexically-independent subject-verb number agreement (NA) on targeted syntactic templates. Our results on nonce sentences suggest that the model generalizes well for simple templates, but fails to perform lexically-independent syntactic generalization when as little as one attractor is present.
  arXiv  Detail & Related papers  (2022-04-14T11:33:15Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.