Tracing the Thought of a Grandmaster-level Chess-Playing Transformer

• URL: http://arxiv.org/abs/2604.10158v1
• Date: Sat, 11 Apr 2026 11:11:06 GMT
• Title: Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
• Abstract summary: We introduce a sparse decomposition framework to interpret Leela Chess Zero's internal computation.<n>We conduct a detailed case study showing that these pathways expose rich, interpretable considerations.<n>To the best of our knowledge, this is the first work to decompose the internal computation of a transformer on both computation and attention modules.
• Score: 62.403498221631715
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: While modern transformer neural networks achieve grandmaster-level performance in chess and other reasoning tasks, their internal computation process remains largely opaque. Focusing on Leela Chess Zero (LC0), we introduce a sparse decomposition framework to interpret its internal computation by decomposing its MLP and attention modules with sparse replacement layers, which capture the primary computation process of LC0. We conduct a detailed case study showing that these pathways expose rich, interpretable tactical considerations that are empirically verifiable. We further introduce three quantitative metrics and show that LC0 exhibits parallel reasoning behavior consistent with the inductive bias of its policy head architecture. To the best of our knowledge, this is the first work to decompose the internal computation of a transformer on both MLP and attention modules for interpretability. Combining sparse replacement layers and causal interventions in LC0 provides a comprehensive understanding of advanced tactical reasoning, offering critical insights into the underlying mechanisms of superhuman systems. Our code is available at https://github.com/JacklE0niden/Leela-SAEs.

Related papers

• Learning through Internalization [34.00603055182392]
  We study internalization processes, by which neural-network-based systems absorb an explicit computational procedure into their own weights.<n>We investigate how transformers internalize the simulation of semiautomata by internalizing chain-of-thought (CoT) tokens.<n>We show that a simplified one-layer transformer provably first learns the target with explicit CoT supervision and then internalizes the autoregressive generation as CoT tokens are progressively removed.
  arXiv  Detail & Related papers  (2026-06-18T21:01:42Z)
• Convolutional Sparse Coding via the Locally Competitive Algorithm on Loihi 2 [0.0]
  We present a Loihi 2 implementation of convolutional sparse coding via the Locally Competitive Algorithm (LCA)<n>To the best of our knowledge, this is the first implementation and benchmark of convolutional LCA on Loihi 2.
  arXiv  Detail & Related papers  (2026-06-07T11:44:40Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• Reasoning is a Modality [4.055765634948606]
  We study abstract reasoning, an ability central to human intelligence.<n>Modern AI systems operate as sequence-of-behavior prediction machines.<n>Humans can explain an action by decoding internal state, while AI systems can produce fluent post-hoc rationalizations.
  arXiv  Detail & Related papers  (2026-01-20T03:37:17Z)
• Bridging Symbolic Control and Neural Reasoning in LLM Agents: The Structured Cognitive Loop [0.0]
  We introduce Structured Cognitive Loop (SCL), a modular architecture that separates agent cognition into five phases: Retrieval, Cognition, Control, Action, and Memory (R-CCAM)<n>At the core of SCL is Soft Symbolic Control, an adaptive governance mechanism that applies symbolic constraints to probabilistic inference.<n>We provide a complete open-source implementation demonstrating the R-CCAM loop architecture, alongside a live GPT-4o-powered travel planning agent.
  arXiv  Detail & Related papers  (2025-11-21T05:19:34Z)
• Provable In-Context Vector Arithmetic via Retrieving Task Concepts [53.685764040547625]
  We show how nonlinear residual transformers trained via gradient descent on cross-entropy loss perform factual-recall ICL tasks via vector arithmetic.<n>These results elucidate the advantages of transformers over static embedding predecessors.
  arXiv  Detail & Related papers  (2025-08-13T13:54:44Z)
• Interpreting Affine Recurrence Learning in GPT-style Transformers [54.01174470722201]
  In-context learning allows GPT-style transformers to generalize during inference without modifying their weights. This paper focuses specifically on their ability to learn and predict affine recurrences as an ICL task. We analyze the model's internal operations using both empirical and theoretical approaches.
  arXiv  Detail & Related papers  (2024-10-22T21:30:01Z)
• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers [54.20763128054692]
  We study how a two-attention-layer transformer is trained to perform ICL on $n$-gram Markov chain data. We prove that the gradient flow with respect to a cross-entropy ICL loss converges to a limiting model.
  arXiv  Detail & Related papers  (2024-09-09T18:10:26Z)
• Language Models as Hierarchy Encoders [22.03504018330068]
  We introduce a novel approach to re-train transformer encoder-based LMs as Hierarchy Transformer encoders (HiTs) Our method situates the output embedding space of pre-trained LMs within a Poincar'e ball with a curvature that adapts to the embedding dimension. We evaluate HiTs against pre-trained LMs, standard fine-tuned LMs, and several hyperbolic embedding baselines.
  arXiv  Detail & Related papers  (2024-01-21T02:29:12Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.