Dissociating Decodability and Causal Use in Bracket-Sequence Transformers

• URL: http://arxiv.org/abs/2604.22128v1
• Date: Fri, 24 Apr 2026 00:26:34 GMT
• Title: Dissociating Decodability and Causal Use in Bracket-Sequence Transformers
• Abstract summary: We find that depth, distance, and top-of-stack signals are all decodable, yet their causal roles diverge.<n>Results, which extend to a templated natural language setting, suggest that even in a controlled setting, decodability alone does not imply causal use.
• Score: 1.8962029954096566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When trained on tasks requiring an understanding of hierarchical structure, transformers have been found to represent this hierarchy in distinct ways: in the geometry of the residual stream, and in stack-like attention patterns maintaining a last-in, first-out ordering. However, it remains unclear whether these representations are causally used or merely decodable. We examine this gap in transformers trained on the Dyck language (a formal language of balanced bracket sequences), where the hierarchical ground truth is explicit. By probing and intervening on the residual stream and attention patterns, we find that depth, distance, and top-of-stack signals are all decodable, yet their causal roles diverge. Specifically, masking attention to the true top-of-stack position causes a sharp drop in long-distance accuracy, while ablating low-dimensional residual stream subspaces has comparatively little effect. These results, which extend to a templated natural language setting, suggest that even in a controlled setting where the relevant hierarchical variables are known, decodability alone does not imply causal use.

Related papers

• A Compositional Theory of Causally Masked Transformers [50.88598486616582]
  We develop a formalization that derives expressivity directly from the model's implemented dynamics.<n>Each attention head updates its own state independently within a layer.<n>A modified form of soft attention supports irreversible, checklist-like state.
  arXiv  Detail & Related papers  (2026-07-29T14:47:19Z)
• Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation [34.50304918655722]
  Transformer representations are known to carry label-level information, but whether they encode semantic operations producing those labels is unclear.<n>We investigate this in Natural Language Inference using controlled premise-hypothesis pairs that differ by a single semantic transformation.
  arXiv  Detail & Related papers  (2026-05-25T07:21:07Z)
• When Rule Violations Are Rare: Chimera Training for Logical Anomaly Detection [56.362776482614976]
  We study anomaly detection in a setting where constraints are given as logical rules over learned visual concepts.<n>We propose a neural rule evaluator that compiles each constraint into a directed acyclic graph and learns feature-aware subtree gates.<n>Across CLEVRER, OpenImages, and VidOR, the resulting evaluator improves rule-level anomaly AUROC over independent-events and same-image semantic-training baselines.
  arXiv  Detail & Related papers  (2026-05-25T02:52:36Z)
• Explicit Dropout: Deterministic Regularization for Transformer Architectures [55.09895958546215]
  Dropout is a widely used regularization technique in deep learning, but its effects are typically realized through masking.<n>We propose a deterministic formulation that expresses dropout as an additive regularizer directly incorporated into the training loss.
  arXiv  Detail & Related papers  (2026-04-22T12:45:51Z)
• From Data Statistics to Feature Geometry: How Correlations Shape Superposition [23.393264014989615]
  A central idea in mechanistic interpretability is that neural networks represent more features than they have dimensions.<n>Superposition has mostly been studied in idealized settings where features are sparse and uncorrelated.<n>We show that superposition is typically understood as introducing interference that must be minimized geometrically and filtered out by non-linearities.
  arXiv  Detail & Related papers  (2026-03-10T17:59:02Z)
• Residual Connections and the Causal Shift: Uncovering a Structural Misalignment in Transformers [9.617245548268437]
  Large Language Models (LLMs) are trained with next-token prediction, implemented in autoregressive Transformers.<n>This creates a subtle misalignment: residual connections tie activations to the current token, while supervision targets the next token.<n>We propose a lightweight residual-path mitigation based on residual attenuation, implemented either as a fixed-layer intervention or as a learnable gating mechanism.
  arXiv  Detail & Related papers  (2026-02-16T14:04:42Z)
• Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution [3.0242762196828448]
  Large Language Models (LLMs) frequently prioritize conflicting in-context information over pre-existing parametric memory.<n>We show that models do not "unlearn" or suppress the magnitude of internal truths but rather employ a mechanism of geometric displacement.
  arXiv  Detail & Related papers  (2026-02-04T06:13:11Z)
• Selective Induction Heads: How Transformers Select Causal Structures In Context [50.09964990342878]
  We introduce a novel framework that showcases transformers' ability to handle causal structures.<n>Our framework varies the causal structure through interleaved Markov chains with different lags while keeping the transition probabilities fixed.<n>This setting unveils the formation of Selective Induction Heads, a new circuit that endows transformers with the ability to select the correct causal structure in-context.
  arXiv  Detail & Related papers  (2025-09-09T23:13:41Z)
• GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features [68.14842693208465]
  GeneralAD is an anomaly detection framework designed to operate in semantic, near-distribution, and industrial settings. We propose a novel self-supervised anomaly generation module that employs straightforward operations like noise addition and shuffling to patch features. We extensively evaluated our approach on ten datasets, achieving state-of-the-art results in six and on-par performance in the remaining.
  arXiv  Detail & Related papers  (2024-07-17T09:27:41Z)
• Transformer Normalisation Layers and the Independence of Semantic Subspaces [17.957364289876548]
  We consider a semantic subspace to be any independent subspace of the latent representation that can fully determine an attention distribution. We show that Pre-Norm, the placement of normalisation layer used by state-of-the-art transformers, violates this ability. We observe a 1% rate of circuit collapse when the norms are artificially perturbed by $lesssim$10%.
  arXiv  Detail & Related papers  (2024-06-25T16:16:38Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• How Transformers Learn Causal Structure with Gradient Descent [44.31729147722701]
  Self-attention allows transformers to encode causal structure. We introduce an in-context learning task that requires learning latent causal structure. We show that transformers trained on our in-context learning task are able to recover a wide variety of causal structures.
  arXiv  Detail & Related papers  (2024-02-22T17:47:03Z)
• Grokking of Hierarchical Structure in Vanilla Transformers [72.45375959893218]
  We show that transformer language models can learn to generalize hierarchically after training for extremely long periods. intermediate-depth models generalize better than both very deep and very shallow transformers.
  arXiv  Detail & Related papers  (2023-05-30T04:34:13Z)
• Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse [11.486545294602697]
  We shed new light on the causes and effects of rank collapse in Transformers. We show that rank collapse of the tokens' representations hinders training by causing the gradients of the queries and keys to vanish.
  arXiv  Detail & Related papers  (2022-06-07T09:07:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.