To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning

• URL: http://arxiv.org/abs/2604.21632v1
• Date: Thu, 23 Apr 2026 12:51:10 GMT
• Title: To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
• Abstract summary: We investigate the ability of decoder-only transformer models to perform abstract symbolic reasoning.<n>We show that a particular representational collapse also has a crucial role: the unembeddings of unseen tokens collapse to nearly the same vector during training.
• Score: 34.300040798165774
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the ability of decoder-only transformer models to perform abstract symbolic reasoning; specifically solving propositional logic reasoning problems given in-context. Previous work demonstrated that models fail to generalize to problems involving variable names that were not observed during training, and it was shown that one reason behind this is the difficulty of copying (or generating) unseen tokens. We show both theoretically and empirically that a particular representational collapse also has a crucial role: the unembeddings (last-layer weights) of unseen tokens collapse to nearly the same vector during training. The collapse makes distinguishing multiple unseen variables difficult for the model (especially when the embedding and unembedding parameters are shared), and provides a mechanistic explanation for the effectiveness of existing heuristic interventions like "active forgetting", which periodically reset the token (un)embeddings. Based on these observations, we devise a combination of techniques, involving a small architecture change facilitating copying, data diversity, and freezing or resetting (un)embeddings, that achieves generalization to unseen tokens. We support our claims with extensive controlled experiments on propositional logic reasoning problems. Beyond synthetic experiments, we also observe evidence of (un)embedding collapse in the open-weight models in the Gemma 3 family, which includes 99 unused tokens reserved for downstream use. Empirically we find that the correlated embeddings of these tokens are a poor initialization for finetuning applications.

Related papers

• Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers [50.98108117044413]
  We show that the choice of a graph-to-token map is a fundamental component of transformer expressivity.<n>We examine three tokenizations that serve as building blocks for many existing graph tokenizations: spectral, random-walk, and adjacency tokenizations.
  arXiv  Detail & Related papers  (2026-05-21T13:32:20Z)
• Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization [15.662726701547896]
  Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors.<n>Despite its prevalence, the characteristics and behaviors of vector quantization in generative models remain largely underexplored.<n>This study systematically investigates the issue of collapses in vector quantization, where collapsed representations are observed across discrete codebook tokens and continuous latent embeddings.
  arXiv  Detail & Related papers  (2026-03-17T18:32:21Z)
• Names Don't Matter: Symbol-Invariant Transformer for Open-Vocabulary Learning [4.288959596387606]
  Current neural architectures lack a principled way to handle interchangeable tokens.<n>We propose a novel Transformer-based mechanism that is provably invariant to the renaming of interchangeable tokens.
  arXiv  Detail & Related papers  (2026-01-30T16:53:01Z)
• In-Context Algebra [25.99258547745488]
  We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their interactions.<n>We devise a new task in which the assignment of symbols to specific algebraic group elements varies from one sequence to another.
  arXiv  Detail & Related papers  (2025-12-18T18:56:50Z)
• Representation Collapsing Problems in Vector Quantization [2.2750239768387255]
  Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. Despite its prevalence, the characteristics and behaviors of vector quantization in generative models remain largely underexplored.
  arXiv  Detail & Related papers  (2024-11-25T16:32:29Z)
• Attuned to Change: Causal Fine-Tuning under Latent-Confounded Shifts [32.989526411946606]
  Adapting to latent-confounded shifts remains a core challenge in modern AI.<n>One practical failure mode arises when fine-tuning pre-trained foundation models on confounded data.<n>We frame causal fine-tuning as an identification problem and pose an explicit causal model that decomposes inputs into low-level spurious features.
  arXiv  Detail & Related papers  (2024-10-18T11:06:23Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Exposing Attention Glitches with Flip-Flop Language Modeling [55.0688535574859]
  This work identifies and analyzes the phenomenon of attention glitches in large language models. We introduce flip-flop language modeling (FFLM), a family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models. We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques.
  arXiv  Detail & Related papers  (2023-06-01T17:44:35Z)
• A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity [71.11795737362459]
  ViTs with self-attention modules have recently achieved great empirical success in many tasks. However, theoretical learning generalization analysis is mostly noisy and elusive. This paper provides the first theoretical analysis of a shallow ViT for a classification task.
  arXiv  Detail & Related papers  (2023-02-12T22:12:35Z)
• An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models [64.87562101662952]
  We show that input tokens are often exchangeable since they already include positional encodings. We establish the existence of a sufficient and minimal representation of input tokens. We prove that attention with the desired parameter infers the latent posterior up to an approximation error.
  arXiv  Detail & Related papers  (2022-12-30T17:59:01Z)
• NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? [61.58261351116679]
  We introduce a two-stage adversarial example generation framework (NaturalAdversaries) for natural language understanding tasks. It is adaptable to both black-box and white-box adversarial attacks based on the level of access to the model parameters. Our results indicate these adversaries generalize across domains, and offer insights for future research on improving robustness of neural text classification models.
  arXiv  Detail & Related papers  (2022-11-08T16:37:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.