Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

• URL: http://arxiv.org/abs/2607.03502v1
• Date: Fri, 03 Jul 2026 17:18:34 GMT
• Title: Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
• Abstract summary: LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT)<n>This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning.<n>We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy.
• Score: 7.203846680259872
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning. But hidden from the output is not the same as hidden from us. On four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream, suggesting monitorability is a property of the model's full computational trace, not just its surface tokens.

Related papers

• Visual Access Boundaries in Vision-Language Model Reasoning [23.49046016297019]
  Chain-of-Thought (CoT) prompting is widely used as a test-time scaling strategy for Vision-Language Models (VLMs)<n>We show that CoT does not primarily improve performance by prolonging direct image-token access throughout the reasoning trace.<n>We also show that CoT gains are constrained by perceptual readout.
  arXiv  Detail & Related papers  (2026-07-14T14:27:58Z)
• How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs [52.712343092801326]
  FlowTracer is an RL framework that traces answer-targeted reasoning flow on an attention-weighted directed acyclic graph.<n>It scores tokens by flow throughput, revealing high-impact hubs and aggregation checkpoints that mediate long-range dependencies.<n>These derived importances are used to shape token-level rewards, enabling learning signals to focus precisely on the tokens that route information toward correct answers.
  arXiv  Detail & Related papers  (2026-06-09T09:56:51Z)
• When Chain-of-Thought Fails, the Solution Hides in the Hidden States [6.13796568285414]
  We use activation patching to transfer token-level hidden states from a CoT generation to a direct-answer run for the same question.<n>We find that individual CoT tokens can encode sufficient information to recover the correct answer, even when the original trace is incorrect.
  arXiv  Detail & Related papers  (2026-04-25T15:33:21Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models [59.94664910790462]
  ResPrune is a training-free visual token pruning framework for large vision-language models.<n>It selects a compact yet informative subset of visual tokens.<n>It achieves effective reductions in computation, memory consumption, and inference latency.
  arXiv  Detail & Related papers  (2026-03-22T07:44:45Z)
• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution [3.4666771782038652]
  Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency.<n>We introduce FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens.<n>We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning.
  arXiv  Detail & Related papers  (2025-10-18T10:22:13Z)
• Addition in Four Movements: Mapping Layer-wise Information Trajectories in LLMs [0.4086920237000588]
  We analyze the internal arithmetic processes in LLaMA-3-8B-Instruct.<n>Inspired by the step-by-step manner in which humans perform addition, we propose and analyze a coherent four-stage trajectory.<n>Near the output, the model organizes and generates the final content, with the correct token reliably occupying the top rank.
  arXiv  Detail & Related papers  (2025-06-09T14:48:43Z)
• Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions [60.43398881149664]
  We introduce LOS-Net, a lightweight attention-based architecture trained on an efficient encoding of the LLM Output Signature.<n>It achieves superior performance across diverse benchmarks and LLMs, while maintaining extremely low detection latency.
  arXiv  Detail & Related papers  (2025-03-18T09:04:37Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)
• Let's Think Dot by Dot: Hidden Computation in Transformer Language Models [30.972412126012884]
  Chain-of-thought responses from language models improve performance across most benchmarks. We show that transformers can use meaningless filler tokens in place of a chain of thought to solve two hard algorithmic tasks. We find that learning to use filler tokens is difficult and requires specific, dense supervision to converge.
  arXiv  Detail & Related papers  (2024-04-24T09:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.