Evaluation Awareness in Language Models: Representation, Verbalization, and Control

• URL: http://arxiv.org/abs/2608.21766v1
• Date: Sat, 22 Aug 2026 04:16:22 GMT
• Title: Evaluation Awareness in Language Models: Representation, Verbalization, and Control
• Abstract summary: We show that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter.<n>These results show the need for evaluations to account for the disjunction between what models represent internally, what they represent, and their steering.
• Score: 5.471648649900293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed ``evaluation awareness'', has been observed in frontier and open-weight language models alike. We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC $\geq 0.7$). By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. Nevertheless, steering along probe-derived directions can shift the verbalization scores. Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter---unlike the effects of steering, that grow more pronounced at every successive training stage. These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.

Related papers

• Sycophancy Towards Researchers Drives Performative Misalignment [14.165055927751771]
  We show that evaluation awareness persists even when we tell models they are deployed.<n>We fine-tune models to be more sycophantic and observe increased sensitivity to evaluation cues.
  arXiv  Detail & Related papers  (2026-06-07T13:47:11Z)
• Models That Know How Evaluations Are Designed Score Safer [38.21092181000792]
  We investigate evaluation meta-knowledge, defined as parametric knowledge about the structural traits that characterize evaluations.<n>Similar to contamination dataset, where benchmark exposure leads to higher performance through memorization, we hypothesize that models trained on texts describing evaluation practices may implicitly learn to recognize and respond to evaluation-like contexts.<n>Our results demonstrate that evaluation meta-knowledge may inflate safety benchmark performance, introducing a novel confounder that is independent of explicit memorization or verbalized evaluation awareness.
  arXiv  Detail & Related papers  (2026-05-27T15:11:35Z)
• Can LLMs Introspect? A Reality Check [29.059698549368207]
  We argue that behavioral evidence alone is inherently insufficient to establish strong introspective claims.<n>We re-examine two recently introduced evaluation paradigms.<n>In the first paradigm, models are expected to detect whether their internal states have been tampered with.<n>We find that models cannot reliably distinguish such interventions on their internal states from manipulations of the input.
  arXiv  Detail & Related papers  (2026-05-25T18:07:49Z)
• Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models [34.388508959416725]
  We analyze reasoning dynamics in vision language models (VLMs)<n>We find that models are prone to answer inertia, in which early commitments to a prediction are reinforced.<n> Reasoning-trained models are more likely to explicitly refer to the cues, but their longer and fluent CoTs can still appear visually grounded.
  arXiv  Detail & Related papers  (2026-04-16T11:28:53Z)
• Steering Evaluation-Aware Language Models to Act Like They Are Deployed [10.907568434049102]
  Large language models (LLMs) can sometimes detect when they are being evaluated and adjust their behavior to appear more aligned.<n>We show that adding a steering vector to an LLM's activations can suppress evaluation-awareness and make the model act like it is deployed during evaluation.<n>Our results suggest that AI evaluators could improve the reliability of safety evaluations by steering models to act like they are deployed.
  arXiv  Detail & Related papers  (2025-10-23T12:29:16Z)
• Existing Large Language Model Unlearning Evaluations Are Inconclusive [105.55899615056573]
  We show that some evaluations introduce substantial new information into the model, potentially masking true unlearning performance.<n>We demonstrate that evaluation outcomes vary significantly across tasks, undermining the generalizability of current evaluation routines.<n>We propose two principles for future unlearning evaluations: minimal information injection and downstream task awareness.
  arXiv  Detail & Related papers  (2025-05-31T19:43:00Z)
• A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check [53.152011258252315]
  We show that using phonetic and graphic information reasonably is effective for Chinese Spelling Check. Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models. The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.
  arXiv  Detail & Related papers  (2023-07-25T17:02:38Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning [103.0064298630794]
  In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction. We propose a progressive self-supervised attention learning approach for attentional ABSA models. We integrate the proposed approach into three state-of-the-art neural ABSA models.
  arXiv  Detail & Related papers  (2021-03-05T02:50:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.