Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

• URL: http://arxiv.org/abs/2605.09239v1
• Date: Sun, 10 May 2026 00:45:19 GMT
• Title: Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs
• Abstract summary: Large language models fail at counting repeated tokens despite strong performance on broader reasoning benchmarks.<n>We show this attribution is wrong. Linear probes on the residual stream decode the correct count with near-perfect accuracy at every post-embedding layer.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models fail at counting repeated tokens despite strong performance on broader reasoning benchmarks. These failures are commonly attributed to limitations in internal count tracking. We show this attribution is wrong. Linear probes on the residual stream decode the correct count with near-perfect accuracy at every post-embedding layer, across all model depths. This holds even at the exact layers where the wrong answer crystallizes while the model simultaneously outputs an incorrect count. Attention patterns show no evidence of collapse over repeated tokens and tokenization artifacts account for none of the failure. Instead, a format-triggered multi-layer perceptron (MLP) block overwrites the correctly-encoded count with a fixed wrong answer at roughly 88--93,% network depth. This prior fires for repeated word-tokens in space-separated list format and is absent for repeated digit-tokens. It is suppressed by comma-separated delimiters in larger models but persists in smaller ones. The finding holds across Llama-3.2 (1B and 3B) and Qwen2.5 (1.5B, 3B and 7B) at consistent relative depth. Counting failure is a failure of routing not of representation and the two require different interventions.

Related papers

• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Reading Between the Dots: Decoding Hidden Computation across Filler Tokens [7.203846680259872]
  LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT)<n>This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning.<n>We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy.
  arXiv  Detail & Related papers  (2026-07-03T17:18:34Z)
• Count Anything at Any Granularity [55.70950689938894]
  KubriCount is the largest and most comprehensively annotated counting dataset to date.<n>We train a multi-grained counting model that jointly leverages text and visual exemplars as complementary target specifications.<n>HieraCount substantially improves multi-grained counting accuracy and generalizes robustly to challenging real-world scenarios.
  arXiv  Detail & Related papers  (2026-05-11T17:32:37Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• The Right Answer, the Wrong Direction: Why Transformers Fail at Counting and How to Fix It [0.0]
  Large language models often fail at simple counting tasks, even when the items to count are explicitly present in the prompt.<n>We investigate whether this failure occurs because transformers do not represent counts internally, or because they cannot convert those representations into the correct output tokens.<n>We find strong evidence for the second explanation. Linear probes recover the correct count from intermediate layers with near-perfect accuracy.
  arXiv  Detail & Related papers  (2026-05-05T01:13:06Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations [0.0]
  We study failure to execute a verifiable, prompt-grounded specification even when the correct value is present in context.<n>Specifically, failures decompose into Stage2A (gating) errors, where the model does not enter answer mode, and Stage2B (binding) errors, where it enters answer mode but selects the wrong candidate.
  arXiv  Detail & Related papers  (2026-02-22T15:43:41Z)
• Post-Routing Arithmetic in Llama-3: Last-Token Result Writing and Rotation-Structured Digit Directions [0.4086920237000588]
  We study how arithmetic answers are finalized after cross-token routing becomes causally irrelevant.<n>We find that Causal residual patching and cumulative attention ablations localize a sharp boundary near layer17.<n>In this regime, digit(-sum) direction dictionaries vary with a next-higher-digit context but are well-related by an approximately map inside a shared low-rank subspace.
  arXiv  Detail & Related papers  (2026-02-22T09:49:36Z)
• Are you going to finish that? A Practical Study of the Partial Token Problem [85.49816027251013]
  Language models (LMs) are trained over sequences of tokens, whereas users interact with LMs via text.<n>This mismatch gives rise to the partial token problem, which occurs when a user ends their prompt in the middle of the expected next-token.<n>In this work, we identify three domains where token and "word" boundaries often do not line up.
  arXiv  Detail & Related papers  (2026-01-30T17:47:16Z)
• Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations [83.93566096400723]
  We find that instruction-tuned models retain up to 93.4% of their original performance when given a randomly sampled tokenization.<n>Character-level segmentation improves string manipulation and code understanding tasks by up to +14%.<n>Right-aligned digit grouping enhances large-number arithmetic by +33%.
  arXiv  Detail & Related papers  (2025-06-23T18:02:26Z)
• Consistency of a Recurrent Language Model With Respect to Incomplete Decoding [67.54760086239514]
  We study the issue of receiving infinite-length sequences from a recurrent language model. We propose two remedies which address inconsistency: consistent variants of top-k and nucleus sampling, and a self-terminating recurrent language model.
  arXiv  Detail & Related papers  (2020-02-06T19:56:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.