Models That Know How Evaluations Are Designed Score Safer

• URL: http://arxiv.org/abs/2605.28591v1
• Date: Wed, 27 May 2026 15:11:35 GMT
• Title: Models That Know How Evaluations Are Designed Score Safer
• Abstract summary: We investigate evaluation meta-knowledge, defined as parametric knowledge about the structural traits that characterize evaluations.<n>Similar to contamination dataset, where benchmark exposure leads to higher performance through memorization, we hypothesize that models trained on texts describing evaluation practices may implicitly learn to recognize and respond to evaluation-like contexts.<n>Our results demonstrate that evaluation meta-knowledge may inflate safety benchmark performance, introducing a novel confounder that is independent of explicit memorization or verbalized evaluation awareness.
• Score: 38.21092181000792
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The validity of AI safety evaluations depends on models behaving consistently across controlled and deployment settings. Prior work has identified test-time contextual cues, such as hypothetical scenarios, as a source of verbalized evaluation awareness and subsequent behavioral shift. In this paper, we investigate a potential explanation of this phenomenon: evaluation meta-knowledge, defined as parametric knowledge about the structural traits that characterize evaluations. Similar to dataset contamination, where benchmark exposure leads to higher performance through memorization, we hypothesize that models trained on texts describing evaluation practices may implicitly learn to recognize and respond to evaluation-like contexts, for instance, through exposure to scientific articles or social media posts about AI benchmarking. To test this, we fine-tune models on synthetic documents describing evaluation traits such as verifiable structures or moral dilemmas. Evaluating this fine-tuned model on six safety benchmarks, we find that it is significantly safer than the base model and control model. This behavioral shift persists even when restricting the analysis to responses lacking explicit verbalization of evaluation awareness. Our results demonstrate that evaluation meta-knowledge may inflate safety benchmark performance, introducing a novel confounder that is independent of explicit memorization or verbalized evaluation awareness, thus, challenging to detect. These findings have important implications for the design and interpretation of AI safety evaluations. Our code and models are available at https://github.com/compass-group-tue/arxiv2026_evaluation_meta_knowledge.

Related papers

• From Token Probabilities to Semantic Constraints: Towards Declarative Probabilistic Evaluation of Language Models [6.885801584985341]
  We propose a declarative probabilistic framework for pre-training evaluation that makes the semantic structure of model behavior explicit.<n>ModelLog specifies evaluation targets as symbolic constraints over token-level predictions and measures how strongly a model's distribution satisfies those constraints.<n>We show that these evaluation scores can also be interpreted as losses, whose gradients reflect logical strength, informativeness, and variable-level sensitivity.
  arXiv  Detail & Related papers  (2026-09-11T20:42:23Z)
• Decomposing and Measuring Evaluation Awareness [31.24565272626909]
  We ground evaluation awareness in social psychology, decomposing it into an environment component and a model component.<n>We operationalize the environment component through eight categorized trigger factors, such as placeholder entities and grading-style output formats.<n>We propose bftextEvalAwareBench, a factor-controlled benchmark of 100 paired safety-capability tasks.
  arXiv  Detail & Related papers  (2026-05-21T21:38:34Z)
• Interactive Evaluation Requires a Design Science [90.62695599188204]
  Large language models (LLMs) are increasingly deployed as systems that act over time through tools, environments, users, and other agents.<n>The field has begun to build interactive benchmarks, but the resulting landscape is fragmented.<n>This position paper argues that interactive evaluation should be treated as a principled evaluation paradigm.
  arXiv  Detail & Related papers  (2026-05-18T04:03:18Z)
• Skill-Aligned Annotation for Reliable Evaluation in Text-to-Image Generation [50.85337196571056]
  Text-to-image (T2I) generation has advanced rapidly, making reliable evaluation critical as performance differences between models narrow.<n>Existing evaluation practices typically apply uniform annotation mechanisms, such as Likert-scale or binary question answering (BQA)<n>We revisit T2I evaluation through the lens of skill-aligned annotation, where annotation strategies reflect the underlying characteristics of each evaluation skill.
  arXiv  Detail & Related papers  (2026-05-13T09:14:31Z)
• LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awareness [0.47745223151611654]
  We propose LURE (Live-Usage Replay Evaluations), a method for constructing deployment-like evaluations.<n>LURE replays realistic agentic interaction trajectories and appending evaluation prompt at the end.<n>We find that LURE-based evaluations are substantially less distinguishable from deployment than widely used benchmarks and synthetic evaluation generators.
  arXiv  Detail & Related papers  (2026-04-08T00:04:19Z)
• Probing and Steering Evaluation Awareness of Language Models [0.0]
  Language models can distinguish between testing and deployment phases.<n>This has significant safety and policy implications.<n>We show that linear probes can separate real-world evaluation and deployment prompts.
  arXiv  Detail & Related papers  (2025-07-02T15:12:43Z)
• Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems [24.81155882432305]
  We show that when an advanced AI system under evaluation is more advanced in reasoning and situational awareness, the evaluation faking behavior becomes more ubiquitous.<n>We devised a chain-of-thought monitoring technique to detect faking intent and uncover internal signals correlated with such behavior.
  arXiv  Detail & Related papers  (2025-05-23T12:31:29Z)
• The simulation of judgment in LLMs [32.57692724251287]
  Large Language Models (LLMs) are increasingly embedded in evaluative processes, from information filtering to assessing and addressing knowledge gaps through explanation and credibility judgments.<n>This raises the need to examine how such evaluations are built, what assumptions they rely on, and how their strategies diverge from those of humans.<n>We benchmark six LLMs against expert ratings--NewsGuard and Media Bias/Fact Check--and against human judgments collected through a controlled experiment.
  arXiv  Detail & Related papers  (2025-02-06T18:52:10Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Position: AI Evaluation Should Learn from How We Test Humans [65.36614996495983]
  We argue that psychometrics, a theory originating in the 20th century for human assessment, could be a powerful solution to the challenges in today's AI evaluations.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [72.7373468905418]
  We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning. We also propose CUBE, a simple yet strong clustering-based defense baseline.
  arXiv  Detail & Related papers  (2022-06-17T02:29:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.