Mechanisms of Introspective Awareness

• URL: http://arxiv.org/abs/2603.21396v1
• Date: Sun, 22 Mar 2026 20:45:34 GMT
• Title: Mechanisms of Introspective Awareness
• Abstract summary: Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept.<n>But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow awarenesss?<n>We investigate these questions in open-source models and establish three main findings.
• Score: 4.6752132340946835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept, a phenomenon cited as evidence of "introspective awareness." But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow heuristics? We investigate these questions in open-source models and establish three main findings. First, introspection is behaviorally robust: detection achieves moderate true positive rates with 0% false positives across diverse prompts. We also find this capability emerges specifically from post-training rather than pretraining. Second, introspection is not reducible to a single linear confound: anomaly detection relies on distributed MLP computation across multiple directions, implemented by evidence carrier and gate features. Third, models possess greater introspective capability than is elicited by default: ablating refusal directions improves detection by 53pp and a trained steering vector by 75pp. Overall, our results suggest that introspective awareness is behaviorally robust, grounded in nontrivial internal anomaly detection, and likely could be substantially improved in future models. Code: https://github.com/safety-research/introspection-mechanisms.

Related papers

• Predicting Future Behaviors in Reasoning Models Enables Better Steering [35.89008737242842]
  Test-time steering controls large reasoning models (LRMs) by intervening on their hidden representations, but it can degrade output quality.<n>We argue that prior steering work implicitly relies on internal features that detect behavior in already generated text.<n>We train activation probes to predict future behavior likelihoods from intermediate reasoning steps.<n>These probes predict the most likely behavior with 64%-91% accuracy, revealing a separate type of internal prediction features.
  arXiv  Detail & Related papers  (2026-06-09T17:49:24Z)
• ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop [55.468404995694975]
  We develop a benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson.<n>We conduct experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts.<n>Unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality.
  arXiv  Detail & Related papers  (2026-05-18T17:59:02Z)
• Decoding the Critique Mechanism in Large Reasoning Models [50.821607345799386]
  Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions.<n>We investigate how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps.<n>We find that despite the error propagating through the chain-of-thought, the model still reaches the correct final answer.
  arXiv  Detail & Related papers  (2026-03-17T10:03:30Z)
• Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection [14.088116748512428]
  Vision-Language-Action (VLA) models have demonstrated strong potential for predicting semantic actions in navigation tasks.<n>They are fundamentally hindered by visual-reasoning hallucinations that lead to trajectory deviations.<n>We show that monitoring a few attention heads within a frozen VLA model can accurately detect path deviations without incurring additional computational overhead.
  arXiv  Detail & Related papers  (2026-03-14T06:26:11Z)
• Backdoor Directions in Vision Transformers [56.382912038371046]
  This paper investigates how Backdoor Attacks are represented within Vision Transformers (ViTs)<n>We identify a specific trigger direction'' in the model's activations that corresponds to the internal representation of the trigger.<n>Using this direction as a diagnostic tool, we trace how backdoor features are processed across layers.
  arXiv  Detail & Related papers  (2026-03-11T14:13:48Z)
• Dissociating Direct Access from Inference in AI Introspection [11.31435294855236]
  Recent work has shown that AI models can introspect.<n>We show that these models detect injected representations via two separable mechanisms.<n>This content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology.
  arXiv  Detail & Related papers  (2026-03-05T17:39:37Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization [56.59356959631999]
  Gated Perception-Reasoning Optimization (GPRO) is a meta-reasoning controller that dynamically routes computation among three decision paths.<n>GPRO substantially improves both accuracy and efficiency, outperforming recent slow-thinking methods.
  arXiv  Detail & Related papers  (2026-01-07T23:05:17Z)
• When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability [0.0]
  Recent work by Anthropic on Mechanistic interpretability claims to understand and control Large Language Models.<n>We conduct an initial stress-test of these claims by replicating their main results with open-source SAEs for Llama 3.1.<n>We find that feature steering exhibits substantial fragility, with sensitivity to layer selection, steering magnitude, and context.
  arXiv  Detail & Related papers  (2026-01-06T14:29:51Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.