Can LLMs Introspect? A Reality Check

• URL: http://arxiv.org/abs/2605.26242v1
• Date: Mon, 25 May 2026 18:07:49 GMT
• Title: Can LLMs Introspect? A Reality Check
• Abstract summary: We argue that behavioral evidence alone is inherently insufficient to establish strong introspective claims.<n>We re-examine two recently introduced evaluation paradigms.<n>In the first paradigm, models are expected to detect whether their internal states have been tampered with.<n>We find that models cannot reliably distinguish such interventions on their internal states from manipulations of the input.
• Score: 29.059698549368207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Can large language models detect and report their own internal states? A number of studies have argued that the answer to this question is yes. We argue, based on lessons from human metacognition research, that this conclusion may be premature: to be convinced of this conclusion we need to distinguish genuine introspection from pattern matching based on surface-level cues. Furthermore, we argue that behavioral evidence alone is inherently insufficient to establish strong introspective claims. We re-examine two recently introduced evaluation paradigms in light of this consideration. In the first paradigm, models are expected to detect whether their internal states have been tampered with. We find that models cannot reliably distinguish such interventions on their internal states from manipulations of the input, suggesting that their success in the original studies reflects their ability to detect anomalies more generally, as opposed to interventions on their internal states in particular. In the second paradigm we examine, models are tasked with predicting labels derived from their own hidden states. Here, we find that classifiers that only have access to the input achieve equivalent performance to the model's own in-context predictions, indicating that the original results do not conclusively demonstrate that the model has privileged access to its internal representations. We further introduce a relabeled control setting, where models cannot rely on the semantics of the task to solve it, and instead must rely on the internal representation; models perform closer to chance on this better-controlled version of the task. Taken together, these results indicate that current evidence is insufficient to establish that LLMs display metacognitive monitoring.

Related papers

• Causality is Key for Interpretability Claims to Generalise [35.833847356014154]
  Interpretability research on large language models (LLMs) has yielded important insights into model behaviour.<n> recurring pitfalls persist: findings that do not generalise, and causal interpretations that outrun the evidence.<n>Pearl's causal hierarchy clarifies what an interpretability study can justify.
  arXiv  Detail & Related papers  (2026-02-18T18:45:04Z)
• Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment [0.3823356975862005]
  We fine-tune GPT-4.1 models sequentially on datasets known to induce and reverse emergent misalignment.<n>Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts.<n>Our findings show that behavioral self-awareness tracks actual alignment states of models, indicating that models can be queried for informative signals about their own safety.
  arXiv  Detail & Related papers  (2026-02-16T14:29:46Z)
• Emergent Introspective Awareness in Large Language Models [2.2458442204933]
  We investigate whether large language models can introspect on their internal states.<n>We find that models can, in certain scenarios, notice the presence of injected concepts and accurately identify them.<n>Claude Opus 4 and 4.1, the most capable models, generally demonstrate the greatest introspective awareness.
  arXiv  Detail & Related papers  (2026-01-05T06:47:41Z)
• Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability [70.4107059502882]
  Training language models with rationales augmentation has been shown to be beneficial in many existing works.<n>We conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance.
  arXiv  Detail & Related papers  (2025-05-30T02:39:37Z)
• A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models [58.32070787537946]
  Chain-of-thought (CoT) reasoning enhances performance of large language models.<n>We present the first comprehensive study of CoT faithfulness in large vision-language models.
  arXiv  Detail & Related papers  (2025-05-29T18:55:05Z)
• Multi-Level Collaboration in Model Merging [56.31088116526825]
  This paper explores the intrinsic connections between model merging and model ensembling.<n>We find that even when previous restrictions are not met, there is still a way for model merging to attain a near-identical and superior performance similar to that of ensembling.
  arXiv  Detail & Related papers  (2025-03-03T07:45:04Z)
• Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models [9.318796743761224]
  Despite growing interest in Theory of Mind (ToM) tasks for evaluating language models (LMs), little is known about how LMs internally represent mental states of self and others.<n>We present the first systematic investigation of belief representations in LMs by probing models across different scales, training regimens, and prompts.<n>Our experiments provide evidence that both model size and fine-tuning substantially improve LMs' internal representations of others' beliefs, which are structured - not mere by-products of spurious correlations - yet brittle to prompt variations.
  arXiv  Detail & Related papers  (2024-06-25T12:51:06Z)
• Predictive Churn with the Set of Good Models [61.00058053669447]
  This paper explores connections between two seemingly unrelated concepts of predictive inconsistency.<n>The first, known as predictive multiplicity, occurs when models that perform similarly produce conflicting predictions for individual samples.<n>The second concept, predictive churn, examines the differences in individual predictions before and after model updates.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning [73.77088902676306]
  We take a closer look at the self-verification abilities of large language models (LLMs) in the context of logical reasoning. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods.
  arXiv  Detail & Related papers  (2023-11-14T07:13:10Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.