Fluid Representations in Reasoning Models

• URL: http://arxiv.org/abs/2602.04843v1
• Date: Wed, 04 Feb 2026 18:34:50 GMT
• Title: Fluid Representations in Reasoning Models
• Authors: Dmitrii Kharlapenko, Alessandro Stolfo, Arthur Conmy, Mrinmaya Sachan, Zhijing Jin,
• Abstract summary: We present a mechanistic analysis of how QwQ-32B processes abstract structural information.<n>We find that QwQ-32B gradually improves its internal representation of actions and concepts during reasoning.
• Score: 91.77876704697779
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning language models, which generate long chains of thought, dramatically outperform non-reasoning language models on abstract problems. However, the internal model mechanisms that allow this superior performance remain poorly understood. We present a mechanistic analysis of how QwQ-32B - a model specifically trained to produce extensive reasoning traces - process abstract structural information. On Mystery Blocksworld - a semantically obfuscated planning domain - we find that QwQ-32B gradually improves its internal representation of actions and concepts during reasoning. The model develops abstract encodings that focus on structure rather than specific action names. Through steering experiments, we establish causal evidence that these adaptations improve problem solving: injecting refined representations from successful traces boosts accuracy, while symbolic representations can replace many obfuscated encodings with minimal performance loss. We find that one of the factors driving reasoning model performance is in-context refinement of token representations, which we dub Fluid Reasoning Representations.

Related papers

• \textit{FocaLogic}: Logic-Based Interpretation of Visual Model Decisions [10.53822145558342]
  FocaLogic is a model-agnostic framework designed to interpret and quantify visual model decision-making through logic-based representations.<n>FocaLogic identifies minimal interpretable subsets of visual regions-termed visual focuses.<n>It translates these visual focuses into precise and compact logical expressions, enabling transparent and structured interpretations.
  arXiv  Detail & Related papers  (2026-01-17T13:28:02Z)
• Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
  We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
  arXiv  Detail & Related papers  (2026-01-07T16:39:34Z)
• Schoenfeld's Anatomy of Mathematical Reasoning by Language Models [56.656180566692946]
  We adopt Schoenfeld's Episode Theory as an inductive, intermediate-scale lens and introduce ThinkARM (Anatomy of Reasoning in Models)<n>ThinkARM explicitly abstracts reasoning traces into functional reasoning steps such as Analysis, Explore, Implement, verify, etc.<n>We show that episode-level representations make reasoning steps explicit, enabling systematic analysis of how reasoning is structured, stabilized, and altered in modern language models.
  arXiv  Detail & Related papers  (2025-12-23T02:44:25Z)
• Stands to Reason: Investigating the Effect of Reasoning on Idiomaticity Detection [2.8330244018167945]
  We examine how reasoning capabilities in Large Language Models affect idiomaticity detection performance.<n>We find the effect of reasoning to be smaller and more varied than expected.<n>For smaller models, producing chain-of-thought (CoT) reasoning increases performance from Math-tuned intermediate models, but not to the levels of the base models.
  arXiv  Detail & Related papers  (2025-08-18T21:17:09Z)
• Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning [9.795934690403374]
  It is still unclear which multi-step reasoning mechanisms are used by language models to solve such tasks.<n>We employ circuit analysis and self-influence functions to evaluate the changing importance of each token throughout the reasoning process.<n>We demonstrate that the underlying circuits reveal a human-interpretable reasoning process used by the model.
  arXiv  Detail & Related papers  (2025-02-13T07:19:05Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Understanding Hidden Computations in Chain-of-Thought Reasoning [0.0]
  Chain-of-Thought (CoT) prompting has significantly enhanced the reasoning abilities of large language models.<n>Recent studies have shown that models can still perform complex reasoning tasks even when the CoT is replaced with filler(hidden) characters.
  arXiv  Detail & Related papers  (2024-12-05T18:43:11Z)
• Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning [11.13665894783481]
  Causal opacity denotes the difficulty in understanding the "hidden" causal structure underlying the decisions of deep neural network (DNN) models.<n>This work introduces Causal Concept Graph Models (Causal CGMs), a class of interpretable models whose decision-making process is causally transparent by design.<n>Our experiments show that Causal CGMs can: (i) match the generalisation performance of causally opaque models, (ii) enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, and (iii) support the analysis of interventional and counterfactual scenarios.
  arXiv  Detail & Related papers  (2024-05-26T10:15:20Z)
• Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism [68.05754701230039]
  We construct a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models.<n>We propose a random matrix-based algorithm to enhance the model's reasoning ability.
  arXiv  Detail & Related papers  (2024-05-24T07:41:26Z)
• Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models [100.53662473219806]
  Diffusion-of-Thought (DoT) is a novel approach that integrates diffusion models with Chain-of-Thought.<n>DoT allows reasoning steps to diffuse over time through a diffusion language model.<n>Our results demonstrate the effectiveness of DoT in multi-digit multiplication, logic, and grade school math problems.
  arXiv  Detail & Related papers  (2024-02-12T16:23:28Z)
• Case-Based Reasoning with Language Models for Classification of Logical Fallacies [3.511369967593153]
  We propose a Case-Based Reasoning method that classifies new cases of logical fallacy. Our experiments indicate that Case-Based Reasoning improves the accuracy and generalizability of language models.
  arXiv  Detail & Related papers  (2023-01-27T17:49:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.