Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

• URL: http://arxiv.org/abs/2606.02907v2
• Date: Wed, 03 Jun 2026 23:53:39 GMT
• Title: Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
• Abstract summary: Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types.<n>We test this by QwensicB on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $$NLI (abductive)
• Score: 16.505918019260964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $α$NLI (abductive). At layer 32 of 40, linear probes achieve 100\% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination $\leq$1.5\%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5\% agreement vs.\ 33.3\% chance), and causal steering with random controls ($n=20$) shows no functional link between geometry and reasoning mode ($p=0.286$). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.

Related papers

• Solution for UCF UrbanTwin V2X-Real Track: Sim-to-Real Urban LiDAR 3D Object Detection [54.03436243843709]
  This report presents a multi-source collaborative training and class-aware fusion framework for Sim2Real 3D detection.<n>The method organizes digital-twin scans, diffusion-redrawn scans, density-stabilized scans, and pedestrian morphology-aligned samples into a unified training pool with complementary roles.
  arXiv  Detail & Related papers  (2026-09-07T15:16:15Z)
• Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard [0.010742675209112621]
  F-ICL is an in-context-learning benchmark for correct inductive inference.<n>We exhaustively enumerate all 1.5 billion programs of length $Lle13$ and compute the Bayes-optimal posterior in closed form.<n>Models answer up to 92% of queries correctly, yet 45 of 46 models yield distributions farther from the optimum than a keystroke reference.
  arXiv  Detail & Related papers  (2026-08-03T01:21:44Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• DeFAb: A Verifiable Benchmark for Defeasible Abduction in Foundation Models [6.628401122676601]
  A rule-based logic solver resolves every instance in our benchmark in under 50 microseconds with 100% accuracy.<n>We introduce DeFAb (Defeasible Abduction Benchmark), a dataset and generation pipeline.
  arXiv  Detail & Related papers  (2026-06-17T00:13:40Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation [0.5451632647096261]
  Cross-Reynolds generalisation in neural PDE solvers remains poorly characterised.<n>We test this hypothesis through ConvAE-Relay, which matches states in a source-trained convolutional autoencoder latent space.<n>Experiments confirm that source-regime dynamics directions remain transferable (cosine similarity 0.84) when matching stays on-manifold.
  arXiv  Detail & Related papers  (2026-05-28T15:49:26Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification [35.08680804423239]
  Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive.<n>We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space.<n>LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch.
  arXiv  Detail & Related papers  (2026-05-08T06:23:58Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• Teaching LLMs Program Semantics via Symbolic Execution Traces [0.7046782561282057]
  We introduce an evaluation framework of 500 C verification tasks built on SV-COMP 2025.<n>We evaluate 14 models across six families and find high overall accuracy masks a critical weakness.<n>Just $sim$3,000 bug traces combined with chain-of-thought reasoning at inference time improve violation detection by over 17 percentage points.
  arXiv  Detail & Related papers  (2026-05-07T13:01:06Z)
• Measuring Representation Robustness in Large Language Models for Geometry [7.743292557234699]
  In geometry, identical problems can be expressed in Euclidean, coordinate, or vector forms.<n>Existing benchmarks report accuracy on fixed formats.<n>We propose GeoRepEval, a representation-aware evaluation framework.
  arXiv  Detail & Related papers  (2026-04-03T11:36:49Z)
• Epistemic Observability in Language Models [0.0]
  We find that models report highest confidence precisely when they are fabricating.<n>We prove, under formal assumptions, that this is not a capability gap but an observational one.<n>We construct a tensor interface that escapes the impossibility by exporting computational byproducts.
  arXiv  Detail & Related papers  (2026-03-20T21:59:34Z)
• DeepPrune: Parallel Scaling without Inter-trace Redundancy [53.62015294143274]
  Over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation.<n>We propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning.<n>Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient.
  arXiv  Detail & Related papers  (2025-10-09T17:24:54Z)
• DDM: A Metric for Comparing 3D Shapes Using Directional Distance Fields [98.15456815880911]
  Existing methods mainly focus on directly establishing the correspondence between two models.<n>We propose DDM, an efficient, effective, robust, and differentiable distance metric for 3D geometry data.<n>As a generic distance metric, DDM has the potential to advance the field of 3D geometric modeling.
  arXiv  Detail & Related papers  (2024-01-18T05:31:53Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.