Probing and Steering Evaluation Awareness of Language Models

• URL: http://arxiv.org/abs/2507.01786v2
• Date: Wed, 09 Jul 2025 08:46:58 GMT
• Title: Probing and Steering Evaluation Awareness of Language Models
• Authors: Jord Nguyen, Khiem Hoang, Carlo Leonardo Attubato, Felix Hofstätter,
• Abstract summary: Language models can distinguish between testing and deployment phases.<n>This has significant safety and policy implications.<n>We show that linear probes can separate real-world evaluation and deployment prompts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Language models can distinguish between testing and deployment phases -- a capability known as evaluation awareness. This has significant safety and policy implications, potentially undermining the reliability of evaluations that are central to AI governance frameworks and voluntary industry commitments. In this paper, we study evaluation awareness in Llama-3.3-70B-Instruct. We show that linear probes can separate real-world evaluation and deployment prompts, suggesting that current models internally represent this distinction. We also find that current safety evaluations are correctly classified by the probes, suggesting that they already appear artificial or inauthentic to models. Our findings underscore the importance of ensuring trustworthy evaluations and understanding deceptive capabilities. More broadly, our work showcases how model internals may be leveraged to support blackbox methods in safety audits, especially for future models more competent at evaluation awareness and deception.

Related papers

• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Sandbagging in a Simple Survival Bandit Problem [3.1764800782234297]
  "Sandbagging" threatens to undermine the integrity of safety evaluations.<n>We develop a simple model of strategic deception in sequential decision-making tasks.<n>We construct a statistical test to distinguish between sandbagging and incompetence from sequences of test scores.
  arXiv  Detail & Related papers  (2025-09-30T13:33:46Z)
• The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs [57.1838332916627]
  Large Language Models (LLMs) have shown remarkable capabilities in Natural Language Processing (NLP)<n>Their widespread deployment has also raised significant safety concerns.<n>LLMs-generated content can exhibit unsafe behaviors such as toxicity, bias, or misinformation, especially in adversarial contexts.
  arXiv  Detail & Related papers  (2025-06-06T05:50:50Z)
• Aurora: Are Android Malware Classifiers Reliable and Stable under Distribution Shift? [51.12297424766236]
  AURORA is a framework to evaluate malware classifiers based on their confidence quality and operational resilience.<n>AURORA is complemented by a set of metrics designed to go beyond point-in-time performance.<n>The fragility in SOTA frameworks across datasets of varying drift suggests the need for a return to the whiteboard.
  arXiv  Detail & Related papers  (2025-05-28T20:22:43Z)
• Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems [24.81155882432305]
  We show that when an advanced AI system under evaluation is more advanced in reasoning and situational awareness, the evaluation faking behavior becomes more ubiquitous.<n>We devised a chain-of-thought monitoring technique to detect faking intent and uncover internal signals correlated with such behavior.
  arXiv  Detail & Related papers  (2025-05-23T12:31:29Z)
• Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods [0.0]
  This literature review consolidates the rapidly evolving field of AI safety evaluations.<n>It proposes a systematic taxonomy around three dimensions: what properties we measure, how we measure them, and how these measurements integrate into frameworks.
  arXiv  Detail & Related papers  (2025-05-08T16:55:07Z)
• Evaluating Frontier Models for Stealth and Situational Awareness [15.820126805686458]
  Recent work has demonstrated the plausibility of frontier AI models scheming.<n>It is important for AI developers to rule out harm from scheming prior to model deployment.<n>We present a suite of scheming reasoning evaluations measuring two types of reasoning capabilities.
  arXiv  Detail & Related papers  (2025-05-02T17:57:14Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains.<n>Existing research predominantly concentrates on the security of general large language models.<n>This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models [59.445672459851274]
  REVAL is a comprehensive benchmark designed to evaluate the textbfREliability and textbfVALue of Large Vision-Language Models.<n>REVAL encompasses over 144K image-text Visual Question Answering (VQA) samples, structured into two primary sections: Reliability and Values.<n>We evaluate 26 models, including mainstream open-source LVLMs and prominent closed-source models like GPT-4o and Gemini-1.5-Pro.
  arXiv  Detail & Related papers  (2025-03-20T07:54:35Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• Trustworthiness for an Ultra-Wideband Localization Service [2.4979362117484714]
  This paper proposes a holistic trustworthiness assessment framework for ultra-wideband self-localization. Our goal is to provide guidance for evaluating a system's trustworthiness based on objective evidence. Our approach guarantees that the resulting trustworthiness indicators correspond to chosen real-world threats.
  arXiv  Detail & Related papers  (2024-08-10T11:57:10Z)
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations [1.0485739694839669]
  Trustworthy capability evaluations are crucial for ensuring the safety of AI systems.<n>Developers of AI systems may have incentives for evaluations to understate the AI's actual capability.<n>In this paper we assess sandbagging capabilities in contemporary language models.
  arXiv  Detail & Related papers  (2024-06-11T15:26:57Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Model evaluation for extreme risks [46.53170857607407]
  Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks.
  arXiv  Detail & Related papers  (2023-05-24T16:38:43Z)
• Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions [48.91284724066349]
  Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education. Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding. We develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of policy evaluation estimates.
  arXiv  Detail & Related papers  (2020-02-10T00:26:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.