Mapping Faithful Reasoning in Language Models

• URL: http://arxiv.org/abs/2510.22362v1
• Date: Sat, 25 Oct 2025 16:48:19 GMT
• Title: Mapping Faithful Reasoning in Language Models
• Abstract summary: We introduce Concept Walk, a framework for tracing how a model's internal stance evolves with respect to a concept direction during reasoning.<n>Unlike surface text, Concept Walk operates in activation space, projecting each reasoning step onto the concept direction learned from contrastive data.<n>We find that in 'easy' cases, perturbed CoTs are quickly ignored, indicating decorative reasoning, whereas in 'hard' cases, perturbations induce sustained shifts in internal activations, consistent with faithful reasoning.
• Score: 1.978797083466893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chain-of-thought (CoT) traces promise transparency for reasoning language models, but prior work shows they are not always faithful reflections of internal computation. This raises challenges for oversight: practitioners may misinterpret decorative reasoning as genuine. We introduce Concept Walk, a general framework for tracing how a model's internal stance evolves with respect to a concept direction during reasoning. Unlike surface text, Concept Walk operates in activation space, projecting each reasoning step onto the concept direction learned from contrastive data. This allows us to observe whether reasoning traces shape outcomes or are discarded. As a case study, we apply Concept Walk to the domain of Safety using Qwen 3-4B. We find that in 'easy' cases, perturbed CoTs are quickly ignored, indicating decorative reasoning, whereas in 'hard' cases, perturbations induce sustained shifts in internal activations, consistent with faithful reasoning. The contribution is methodological: Concept Walk provides a lens to re-examine faithfulness through concept-specific internal dynamics, helping identify when reasoning traces can be trusted and when they risk misleading practitioners.

Related papers

• MUtE: A Dual Framework for Concept Erasure and Counterfactual Interventions [0.0]
  Erasing concept-specific information from representations has been proven useful for mitigating bias or interpreting model decisions.<n>We revisit the optimal bounds of concept erasure to derive a novel class of erasure functions that naturally induce a deterministic, dual counterfactual mapping.
  arXiv  Detail & Related papers  (2026-09-10T08:47:11Z)
• On the Faithfulness of Post-Hoc Concept Bottleneck Models [7.774981803358894]
  We study Post-Hoc Concept Bottleneck Models (post-hoc CBMs)<n>CBMs project latent features onto interpretable concept spaces using auxiliary datasets or vision-language models.<n>For example, random concept projections can achieve competitive accuracy despite being semantically meaningless.
  arXiv  Detail & Related papers  (2026-06-29T16:02:29Z)
• Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs [54.16324124242172]
  Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models.<n>We identify a previously overlooked optimization pathology in existing latent visual reasoning methods.<n>We show that inference-time latent optimization, without any parameter updates, effectively unleashes the suppressed reasoning capacity of visual latents.
  arXiv  Detail & Related papers  (2026-05-04T15:36:12Z)
• Navigating the Conceptual Multiverse [8.506067537110214]
  We build and evaluate the conceptual multiverse, an interactive system that represents conceptual decisions.<n>We develop a general verification framework that enforces properties of good decision structures.<n>Across three calibrated domains, the conceptual multiverse helped participants develop a working map of a problem.
  arXiv  Detail & Related papers  (2026-04-20T05:12:59Z)
• Can VLMs Reason Robustly? A Neuro-Symbolic Investigation [24.000218548502875]
  Vision-Language Models (VLMs) have been applied to a wide range of reasoning tasks.<n>It remains unclear whether they can reason robustly under distribution shifts.<n>We propose a neuro-symbolic method that combines VLM-based concept recognition with circuit-based symbolic reasoning.
  arXiv  Detail & Related papers  (2026-03-25T02:48:42Z)
• Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking [11.763473690046721]
  Reasoning-augmented vision language models generate explicit chains of thought that promise greater capability and transparency.<n>Models may reach correct answers via visually unfaithful intermediate steps, or reason faithfully yet fail on the final prediction.<n>We introduce the visual faithfulness of reasoning chains as a distinct evaluation dimension, focusing on whether the perception steps of a reasoning chain are grounded in the image.
  arXiv  Detail & Related papers  (2025-12-13T07:04:42Z)
• Artemis: Structured Visual Reasoning for Perception Policy Learning [64.57381337070616]
  Empirical observations indicate that purely linguistic intermediate reasoning often reduces performance on perception tasks.<n>We introduce Artemis, a perception-policy learning framework that performs structured proposal-based reasoning.
  arXiv  Detail & Related papers  (2025-12-01T18:45:30Z)
• FaCT: Faithful Concept Traces for Explaining Neural Network Decisions [56.796533084868884]
  Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge.<n>We put emphasis on the faithfulness of concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations.<n>Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced.
  arXiv  Detail & Related papers  (2025-10-29T13:35:46Z)
• ActivationReasoning: Logical Reasoning in Latent Activation Spaces [43.17973499652433]
  Large language models (LLMs) excel at generating fluent text, but their internal reasoning remains opaque and difficult to control.<n>We introduce ActivationReasoning (AR), a framework that embeds explicit logical reasoning into the latent space of LLMs.<n>AR scales robustly with reasoning complexity, generalizes to abstract and context-sensitive tasks, and transfers across model backbones.
  arXiv  Detail & Related papers  (2025-10-21T00:21:04Z)
• A Survey on Latent Reasoning [100.54120559169735]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities.<n>CoT reasoning that verbalizes intermediate steps limits the model's expressive bandwidth.<n>Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state.
  arXiv  Detail & Related papers  (2025-07-08T17:29:07Z)
• How do Transformers Learn Implicit Reasoning? [67.02072851088637]
  We study how implicit multi-hop reasoning emerges by training transformers from scratch in a controlled symbolic environment.<n>We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures.
  arXiv  Detail & Related papers  (2025-05-29T17:02:49Z)
• Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space [62.54887038032942]
  We introduce Soft Thinking, a training-free method that emulates human-like "soft" reasoning by generating soft, abstract concept tokens.<n>These concept tokens are created by the probability-weighted mixture of token embeddings, which form the continuous concept space.<n>In essence, each generated concept token encapsulates multiple meanings from related discrete tokens, implicitly exploring various reasoning paths to converge.
  arXiv  Detail & Related papers  (2025-05-21T17:29:15Z)
• Take A Step Back: Rethinking the Two Stages in Visual Reasoning [57.16394309170051]
  This paper revisits visual reasoning with a two-stage perspective. It is more efficient to implement symbolization via separated encoders for different data domains while using a shared reasoner. The proposed two-stage framework achieves impressive generalization ability on various visual reasoning tasks.
  arXiv  Detail & Related papers  (2024-07-29T02:56:19Z)
• Calibrating Reasoning in Language Models with Internal Consistency [18.24350001344488]
  Large language models (LLMs) have demonstrated impressive capabilities in various reasoning tasks.<n>LLMs often generate text with obvious mistakes and contradictions.<n>In this work, we investigate reasoning in LLMs through the lens of internal representations.
  arXiv  Detail & Related papers  (2024-05-29T02:44:12Z)
• Contrastive Chain-of-Thought Prompting [74.10511560147293]
  We propose contrastive chain of thought to enhance language model reasoning. Compared to the conventional chain of thought, our approach provides both valid and invalid reasoning demonstrations. Our experiments on reasoning benchmarks demonstrate that contrastive chain of thought can serve as a general enhancement of chain-of-thought prompting.
  arXiv  Detail & Related papers  (2023-11-15T18:54:01Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• Interpretable Neural-Symbolic Concept Reasoning [7.1904050674791185]
  Concept-based models aim to address this issue by learning tasks based on a set of human-understandable concepts. We propose the Deep Concept Reasoner (DCR), the first interpretable concept-based model that builds upon concept embeddings.
  arXiv  Detail & Related papers  (2023-04-27T09:58:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.