Training Introspective Behavior: Fine-Tuning Induces Reliable Internal State Detection in a 7B Model

• URL: http://arxiv.org/abs/2511.21399v1
• Date: Wed, 26 Nov 2025 13:49:43 GMT
• Title: Training Introspective Behavior: Fine-Tuning Induces Reliable Internal State Detection in a 7B Model
• Abstract summary: Lindsey (2025) investigates introspective awareness in language models through four experiments.<n>We focus on the first of these experiments -- self-report of injected "thoughts"<n>We show that at least one component of introspective behavior can be directly induced, offering a pathway to built-in AI transparency.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lindsey (2025) investigates introspective awareness in language models through four experiments, finding that models can sometimes detect and identify injected activation patterns -- but unreliably (~20% success in the best model). We focus on the first of these experiments -- self-report of injected "thoughts" -- and ask whether this capability can be directly trained rather than waiting for emergence. Through fine-tuning on transient single-token injections, we transform a 7B parameter model from near-complete failure (0.4% accuracy, 6.7% false positive rate) to reliable detection (85% accuracy on held-out concepts at α=40, 0% false positives). Our model detects fleeting "thoughts" injected at a single token position, retains that information, and reports the semantic content across subsequent generation steps. On this task, our trained model satisfies three of Lindsey's criteria: accuracy (correct identification), grounding (0/60 false positives), and internality (detection precedes verbalization). Generalization to unseen concept vectors (7.5pp gap) demonstrates the model learns a transferable skill rather than memorizing specific vectors, though this does not establish metacognitive representation in Lindsey's sense. These results address an open question raised by Lindsey: whether "training for introspection would help eliminate cross-model differences." We show that at least one component of introspective behavior can be directly induced, offering a pathway to built-in AI transparency.

Related papers

• Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models [56.88565331713778]
  reasoning-oriented training can make traces look more deliberative without amplifying the behaviors most tied to model correctness.<n>We quantify this mismatch with Behavioral Lift, a metric that measures how much correctness changes when a behavior is present versus absent in a model's reasoning trace.<n>We find evidence for an Amplification-Lift Gap, in which thinking models strongly amplify self-awareness, hypothesis testing, and uncertainty acknowledgment.
  arXiv  Detail & Related papers  (2026-08-13T20:37:59Z)
• "Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms [2.3883252067956118]
  We show that existing trained model organisms often fail testbeds where models verifiably believe the opposite of what they say.<n>We address this with 13 reasoning model organisms whose hidden beliefs are verified in chain-of-thought.<n>On prompted lying, across 31 open-weight models spanning 2B to 1T parameters, all four detectors show positive scaling with model capability.
  arXiv  Detail & Related papers  (2026-06-10T19:21:12Z)
• Understanding and Mitigating Premature Confidence for Better LLM Reasoning [76.16007941549857]
  Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps.<n>We find such a signal in how the model's confidence evolves during reasoning.<n>We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early.
  arXiv  Detail & Related papers  (2026-05-23T04:42:45Z)
• What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models [0.0]
  Single-prompt accuracy is the dominant way to benchmark language models, but it can miss reliability failures that matter.<n>We evaluate a 15-model open-weight corpus, with the main reliability analyses focused on 10 instruct models across five classification and reasoning benchmarks.<n>We find three broad results. First, evaluation design can materially change the conclusion.<n>Second, confidence signals are fragile. On MMLU-Pro, every primary model verbally reports confidence substantially above both its accuracy and its token-probability confidence on the same rows, and verbal parse rate can collapse for a single model on a single prompt variant.
  arXiv  Detail & Related papers  (2026-05-03T20:05:08Z)
• Latent Introspection: Models Can Detect Prior Concept Injections [0.11405605998490648]
  We uncover a latent capacity for introspection in a Qwen 32B model.<n>The model can detect when concepts have been injected into its earlier context and identify which concept was injected.
  arXiv  Detail & Related papers  (2026-02-23T16:39:42Z)
• Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities [15.59200865541989]
  We introduce Split Personality Training (SPT) to fine-tune a second honest persona'' into parameters that remain inactive during normal operation.<n>SPT achieves 96% overall accuracy, whereas Anthropic reports near 0% accuracy.
  arXiv  Detail & Related papers  (2026-02-05T10:45:48Z)
• Do I Really Know? Learning Factual Self-Verification for Hallucination Reduction [14.310806623700037]
  We propose a training-time framework that teaches large language models to reason about factual uncertainty through consistency-based self-verification.<n>Across multiple model families and scales, VeriFY reduces factual hallucination rates by 9.7 to 53.3 percent, with only modest reductions in recall.<n>The source code, training data, and trained model checkpoints will be released upon acceptance.
  arXiv  Detail & Related papers  (2026-02-02T12:15:50Z)
• Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Models [71.44858461725893]
  Given a model fine-tuned by an untrusted third party, determining whether the model has been injected with a backdoor is a critical and challenging problem.<n>Existing detection methods usually rely on prior knowledge of training dataset, backdoor triggers and targets.<n>We introduce Assimilation Matters in DETection (AMDET), a novel model-level detection framework that operates without any such prior knowledge.
  arXiv  Detail & Related papers  (2025-11-29T06:20:00Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)
• No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes [2.6550928535945872]
  We train linear probes to predict whether a model's forthcoming answer will be correct.<n>Across three open-source model families, projections on this "in-advance correctness direction" trained on generic trivia questions predict success in distribution.<n>For models responding "I don't know", doing so strongly correlates with the probe score, indicating that the same direction also captures confidence.
  arXiv  Detail & Related papers  (2025-09-12T18:09:55Z)
• Probing for Arithmetic Errors in Language Models [86.8227317662622]
  Internal activations in language models can be used to detect arithmetic errors.<n>We show that simple probes can accurately decode both the model's predicted output and the correct answer from hidden states.<n>We train lightweight error detectors that predict model correctness with over 90% accuracy.
  arXiv  Detail & Related papers  (2025-07-16T16:27:50Z)
• Lie Detector: Unified Backdoor Detection via Cross-Examination Framework [68.45399098884364]
  We propose a unified backdoor detection framework in the semi-honest setting.<n>Our method achieves superior detection performance, improving accuracy by 5.4%, 1.6%, and 11.9% over SoTA baselines.<n> Notably, it is the first to effectively detect backdoors in multimodal large language models.
  arXiv  Detail & Related papers  (2025-03-21T06:12:06Z)
• Pre-training and Diagnosing Knowledge Base Completion Models [58.07183284468881]
  We introduce and analyze an approach to knowledge transfer from one collection of facts to another without the need for entity or relation matching. The main contribution is a method that can make use of large-scale pre-training on facts, which were collected from unstructured text. To understand the obtained pre-trained models better, we then introduce a novel dataset for the analysis of pre-trained models for Open Knowledge Base Completion.
  arXiv  Detail & Related papers  (2024-01-27T15:20:43Z)
• Discovering Latent Knowledge in Language Models Without Supervision [72.95136739040676]
  Existing techniques for training language models can be misaligned with the truth. We propose directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models.
  arXiv  Detail & Related papers  (2022-12-07T18:17:56Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.