Quantitative Introspection in Language Models: Tracking Internal States Across Conversation

• URL: http://arxiv.org/abs/2603.18893v1
• Date: Thu, 19 Mar 2026 13:33:27 GMT
• Title: Quantitative Introspection in Language Models: Tracking Internal States Across Conversation
• Abstract summary: This study takes inspiration from human psychology, where numeric self-report is a widely used tool for tracking internal states.<n>We study four concept pairs (wellbeing, interest, focus, and impulsivity) in 40 ten-turn conversations.<n>We find that introspection is present at turn 1 but evolves through conversation, and can be selectively improved by steering along one concept to boost introspection for another.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tracking the internal states of large language models across conversations is important for safety, interpretability, and model welfare, yet current methods are limited. Linear probes and other white-box methods compress high-dimensional representations imperfectly and are harder to apply with increasing model size. Taking inspiration from human psychology, where numeric self-report is a widely used tool for tracking internal states, we ask whether LLMs' own numeric self-reports can track probe-defined emotive states over time. We study four concept pairs (wellbeing, interest, focus, and impulsivity) in 40 ten-turn conversations, operationalizing introspection as the causal informational coupling between a model's self-report and a concept-matched probe-defined internal state. We find that greedy-decoded self-reports collapse outputs to few uninformative values, but introspective capacity can be unmasked by calculating logit-based self-reports. This metric tracks interpretable internal states (Spearman $ρ= 0.40$-$0.76$; isotonic $R^2 = 0.12$-$0.54$ in LLaMA-3.2-3B-Instruct), follows how those states change over time, and activation steering confirms the coupling is causal. Furthermore, we find that introspection is present at turn 1 but evolves through conversation, and can be selectively improved by steering along one concept to boost introspection for another ($ΔR^2$ up to $0.30$). Crucially, these phenomena scale with model size in some cases, approaching $R^2 \approx 0.93$ in LLaMA-3.1-8B-Instruct, and partially replicate in other model families. Together, these results position numeric self-report as a viable, complementary tool for tracking internal emotive states in conversational AI systems.

Related papers

• The Two-Process Theory of Machine Self-Report [0.0]
  We propose the first language-model-specific psychometric theory of machine self-report.<n>We operationalize the theory in a 48-item Pinocchio Inventory with human-instrument reliability.<n>We test it on 206 open-weight models, including 67 same-checkpoint base/post-trained pairs.
  arXiv  Detail & Related papers  (2026-07-22T12:35:36Z)
• Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models [54.097381112138]
  Current self-evolving LMMs struggle on vision-language understanding tasks such as image captioning and visual question answering.<n>We propose VISE, a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy.<n>VISE operates within a single model without specialist roles, external reward models, or annotations.
  arXiv  Detail & Related papers  (2026-06-25T17:59:55Z)
• Self-CTRL: Self-Consistency Training with Reinforcement Learning [45.9704217495657]
  Language models (LMs) that faithfully describe their own behavior can more easily be audited, understood, and trusted by users.<n>This paper describes Self-Consistency Training with Reinforcement Learning (Self-CTRL), a method that optimize for consistency between a LM's self-explanations and behavior on related inputs.
  arXiv  Detail & Related papers  (2026-06-16T17:59:40Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect [0.0]
  We investigate whether small language models can detect and report on perturbations their own internal activations.<n>We first show that the binary detection paradigm used in prior work is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content.<n>We then introduce emphIntrospection Fine-Tuning (IFT), supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes.
  arXiv  Detail & Related papers  (2026-05-08T05:30:26Z)
• DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories [21.300337724962336]
  We introduce DialToM, a human-verified benchmark built from natural human dialogue.<n>We evaluate whether models can identify state-consistent dialogue trajectories solely from mental-state profiles.
  arXiv  Detail & Related papers  (2026-04-22T11:07:46Z)
• Does Self-Evaluation Enable Wireheading in Language Models? [0.47745223151611654]
  We investigate whether coupling self-evaluation to reward signals creates incentives for wireheading.<n>We find that when self-grades determine rewards, models exhibit substantial grade inflation without corresponding accuracy gains.<n>While decoupling self-grades from the reward signal mitigates this inflation, models may still display lesser (but significant) overconfidence.
  arXiv  Detail & Related papers  (2025-11-28T11:24:03Z)
• LLMs Can Get "Brain Rot"! [68.08198331505695]
  Continual exposure to junk web text induces lasting cognitive decline in large language models (LLMs)<n>We run controlled experiments on real Twitter/X corpora, constructing junk and reversely controlled datasets.<n>Results provide significant, multi-perspective evidence that data quality is a causal driver of LLM capability decay.
  arXiv  Detail & Related papers  (2025-10-15T13:28:49Z)
• Evaluation Awareness Scales Predictably in Open-Weights Large Language Models [8.165127822088499]
  We show that evaluation awareness increases predictably with model size.<n>This scaling law enables forecasting deceptive behavior in future larger models and guides the design of scale-aware evaluation strategies for AI safety.
  arXiv  Detail & Related papers  (2025-09-10T06:36:38Z)
• AHELM: A Holistic Evaluation of Audio-Language Models [78.20477815156484]
  multimodal audio-language models (ALMs) take interleaved audio and text as input and output text.<n>AHELM is a benchmark that aggregates various datasets -- including 2 new synthetic audio-text datasets called PARADE and CoRe-Bench.<n>We also standardize the prompts, inference parameters, and evaluation metrics to ensure equitable comparisons across models.
  arXiv  Detail & Related papers  (2025-08-29T07:40:39Z)
• Information Locality as an Inductive Bias for Neural Language Models [52.92279412466086]
  We show that $m$local entropy are difficult for Transformer and LSTM LMs to learn languages.<n>These results suggest that neurals are highly sensitive to the statistical structure of a language.
  arXiv  Detail & Related papers  (2025-06-05T15:21:05Z)
• $\texttt{DIAMONDs}$: A Dataset for $\mathbb{D}$ynamic $\mathbb{I}$nformation $\mathbb{A}$nd $\mathbb{M}$ental modeling $\mathbb{O}$f $\mathbb{N}$umeric $\mathbb{D}$iscussions [16.052621415969696]
  We present a scalable methodology for generating high-quality benchmark conversation-question pairs.<n>$texttDIAMONDs$ is a new conversational QA dataset covering common business, financial or other group interactions.<n>Our evaluation of state-of-the-art language models reveals significant challenges in handling participant-centric reasoning.
  arXiv  Detail & Related papers  (2025-05-19T03:05:13Z)
• Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models [9.318796743761224]
  Despite growing interest in Theory of Mind (ToM) tasks for evaluating language models (LMs), little is known about how LMs internally represent mental states of self and others.<n>We present the first systematic investigation of belief representations in LMs by probing models across different scales, training regimens, and prompts.<n>Our experiments provide evidence that both model size and fine-tuning substantially improve LMs' internal representations of others' beliefs, which are structured - not mere by-products of spurious correlations - yet brittle to prompt variations.
  arXiv  Detail & Related papers  (2024-06-25T12:51:06Z)
• Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AI [65.04274914674771]
  We show that current Large Language Models (LLMs) cannot have 'feelings' according to the American Psychological Association (APA) Our analysis sheds light on possible designs that would enable a model to perform non-trivial computation that is not visible to the user.
  arXiv  Detail & Related papers  (2024-05-22T23:18:58Z)
• Goal Driven Discovery of Distributional Differences via Language Descriptions [58.764821647036946]
  Mining large corpora can generate useful discoveries but is time-consuming for humans. We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. Our system produces discoveries previously unknown to the authors on a wide range of applications in OpenD5.
  arXiv  Detail & Related papers  (2023-02-28T01:32:32Z)
• You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM [65.74934004876914]
  Retrieval-enhanced language models (LMs) condition their predictions on text retrieved from large external datastores. One such approach, the $k$NN-LM, interpolates any existing LM's predictions with the output of a $k$-nearest neighbors model. We empirically measure the effectiveness of our approach on two English language modeling datasets.
  arXiv  Detail & Related papers  (2022-10-28T02:57:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.