Strangers to Themselves: What Language Models Say About Themselves Is Generic

• URL: http://arxiv.org/abs/2609.09899v1
• Date: Wed, 09 Sep 2026 08:56:02 GMT
• Title: Strangers to Themselves: What Language Models Say About Themselves Is Generic
• Abstract summary: Language models can fluently describe how they would behave.<n>Is that description actually about the model speaking?<n>We turn self-knowledge into a prediction test.
• Score: 0.33985395340995606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models can fluently describe how they would behave: whether they would cave to pushback, misuse a tool, or lie under pressure. Is that description actually about the model speaking? We turn self-knowledge into a prediction test. Across nine behavioral evaluations, we measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. We find that: (i) Direct self-report is weak (r = +0.04), and even showing the model the exact items only raises prediction to +0.24. Crucially, the same item-informed question about "capable AI agents in general" does just as well (+0.28), while other models' answers about themselves predict the target model at least as well as its own. (ii) Frontier scale does not detectably change this pattern: any gains in prediction are not self-specific, and are consistent with a better theory of how AI assistants behave rather than better self-knowledge. (iii) First-person framing does have one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic agent. (iv) Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted and the gains do not transfer broadly. The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model.

Related papers

• An Empirical Study of Counterfactual Self-Explanations in LLMs [13.093234754814851]
  Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior.<n>We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes.<n>Our results show that model scale is the strongest determinant of explanation quality.
  arXiv  Detail & Related papers  (2026-09-15T12:47:22Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior [11.616524876789624]
  LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood.<n>We introduce Normalized Simulata Gainbility (NSG), a metric based on the idea that a faithful explanation should allow an observer to learn a model's decision-making criteria.<n>We find self-explanations substantially improve prediction of model behavior (11-37% NSG)
  arXiv  Detail & Related papers  (2026-02-02T18:54:51Z)
• Know Thyself? On the Incapability and Implications of AI Self-Recognition [22.582593406983907]
  Self-recognition is a crucial metacognitive capability for AI systems, relevant not only for psychological analysis but also for safety.<n>We introduce a systematic evaluation framework that can be easily applied and updated.<n>We measure how well 10 contemporary larger language models (LLMs) can identify their own generated text versus text from other models.
  arXiv  Detail & Related papers  (2025-10-03T18:00:01Z)
• Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors [61.92704516732144]
  We show that the most robust features for correctness prediction are those that play a distinctive causal role in the model's behavior.<n>We propose two methods that leverage causal mechanisms to predict the correctness of model outputs.
  arXiv  Detail & Related papers  (2025-05-17T00:31:39Z)
• Great Models Think Alike and this Undermines AI Oversight [47.7725284401918]
  We study how model similarity affects both aspects of AI oversight.<n>We propose CAPA: a metric for LM similarity based on overlap in model mistakes.<n>Our work underscores the importance of reporting and correcting for model similarity.
  arXiv  Detail & Related papers  (2025-02-06T18:56:01Z)
• SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs [72.06808538971487]
  We test whether large language models (LLMs) can implicitly apply a "theory of mind" (ToM) to predict behavior. We create a new dataset, SimpleTom, containing stories with three questions that test different degrees of ToM reasoning. To our knowledge, SimpleToM is the first dataset to explore downstream reasoning requiring knowledge of mental states in realistic scenarios.
  arXiv  Detail & Related papers  (2024-10-17T15:15:00Z)
• Limitations of Agents Simulated by Predictive Models [1.6649383443094403]
  We outline two structural reasons for why predictive models can fail when turned into agents. We show that both of those failures are fixed by including a feedback loop from the environment. Our treatment provides a unifying view of those failure modes, and informs the question of why fine-tuning offline learned policies with online learning makes them more effective.
  arXiv  Detail & Related papers  (2024-02-08T17:08:08Z)
• Rationalizing Predictions by Adversarial Information Calibration [65.19407304154177]
  We train two models jointly: one is a typical neural model that solves the task at hand in an accurate but black-box manner, and the other is a selector-predictor model that additionally produces a rationale for its prediction. We use an adversarial technique to calibrate the information extracted by the two models such that the difference between them is an indicator of the missed or over-selected features.
  arXiv  Detail & Related papers  (2023-01-15T03:13:09Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.