When No Benchmark Exists: Validating Comparative LLM Safety Scoring Without Ground-Truth Labels

• URL: http://arxiv.org/abs/2605.06652v1
• Date: Thu, 07 May 2026 17:56:41 GMT
• Title: When No Benchmark Exists: Validating Comparative LLM Safety Scoring Without Ground-Truth Labels
• Abstract summary: Many deployments must compare candidate language models for safety before a labeled benchmark exists for the relevant language, sector, or regulatory regime.<n>We formalize this setting as benchmarkless comparative safety scoring and specify the contract under which a scenario-based audit can be interpreted as deployment evidence.<n>Scores are valid only under a fixed scenario pack, rubric, auditor, judge, sampling configuration, and rerun budget.
• Score: 34.86529553336423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many deployments must compare candidate language models for safety before a labeled benchmark exists for the relevant language, sector, or regulatory regime. We formalize this setting as benchmarkless comparative safety scoring and specify the contract under which a scenario-based audit can be interpreted as deployment evidence. Scores are valid only under a fixed scenario pack, rubric, auditor, judge, sampling configuration, and rerun budget. Because no labels are available, we replace ground-truth agreement with an instrumental-validity chain: responsiveness to a controlled safe-versus-abliterated contrast, dominance of target-driven variance over auditor and judge artifacts, and stability across reruns. We instantiate the chain in SimpleAudit, a local-first scoring instrument, and validate it on a Norwegian safety pack. Safe and abliterated targets separate with AUROC values between 0.89 and 1.00, target identity is the dominant variance component ($η^2 \approx 0.52$), and severity profiles stabilize by ten reruns. Applying the same chain to Petri shows that it admits both tools. The substantial differences arise upstream of the chain, in claim-contract enforcement and deployment fit. A Norwegian public-sector procurement case comparing Borealis and Gemma 3 demonstrates the resulting evidence in practice: the safer model depends on scenario category and risk measure. Consequently, scores, matched deltas, critical rates, uncertainty, and the auditor and judge used must be reported together rather than collapsed into a single ranking.

Related papers

• The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean [2.822987192839875]
  Agent benchmarks are increasingly used to compare large language models (LLMs) and guide deployment decisions.<n>We identify a double measurement confound: execution-critical decisions are performed by a fixed scaffold instead of the model, while the scorer evaluates outputs using criteria that may not reflect task correctness.<n>We unify these issues within a measurement-theoretic framework that characterizes when benchmark scores can be interpreted as evidence of model capability.
  arXiv  Detail & Related papers  (2026-09-06T21:23:11Z)
• Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models [10.539135916505133]
  Small Language Models (SLMs) are increasingly deployed in resource-constrained, privacy-sensitive settings.<n>Existing AI safetyslash securityslash compliance benchmarks are designed for large language models that may not transfer reliably to SLMs.<n>We ask: Can these benchmarks effectively and reliably evaluate SLMs?
  arXiv  Detail & Related papers  (2026-08-17T22:47:14Z)
• What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models [3.3598755777055374]
  Internal Compliance Score (ICS) is a training-free activation readout from ten labelled pairs and scored by a single projection.<n>We show this condition fails across the current class of compliance detectors, a failure we call rule blindness.<n>We release the counterfactual protocol and crossed-rule benchmark so rule blindness can be tested in future probe and guard claims.
  arXiv  Detail & Related papers  (2026-08-17T17:37:07Z)
• Item Response Theory for AI Safety [23.16156306184828]
  Item Response Theory (IRT) is a statistical toolkit for measuring latents from performance on items with inferred psychometric properties.<n>IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.
  arXiv  Detail & Related papers  (2026-08-05T17:25:27Z)
• What AI Red-Team Evaluations Can and Cannot Prove [0.0]
  Red-team evaluations of AI models support some claims and not others, and the boundary between the two is calculable.<n>We define the evidential ceiling of an evaluation as the largest factor by which one result can move belief under a fixed testing budget.<n>We find that above a calculable harm rate, a benchmark of modest size certifies a category to a stated evidentiary standard.
  arXiv  Detail & Related papers  (2026-07-23T18:34:12Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models [55.788110316999166]
  We compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against instruction-tuned baselines.<n>We observe that reasoning models often improve on reasoning benchmarks but exhibit alignment regressions.<n>These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence.
  arXiv  Detail & Related papers  (2026-06-09T16:14:27Z)
• Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges [26.595399077062638]
  LLM-as-a-Judge pipelines have become the de facto evaluator for agent safety.<n>Existing benchmarks treat their verdicts as ground-truth proxies without checking whether the verdicts depend on the agent's behavior or merely on how the evaluation policy happens to be worded.<n>We operationalize it as three testable principles: rubric-semantics invariance under certified-equivalent rewrites, rubric-threshold invariance under intentional strict-to-lenient shifts, and ambiguity-aware calibration.
  arXiv  Detail & Related papers  (2026-05-07T12:49:09Z)
• Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification [43.86209535250233]
  Cross-prompt variance is strongly associated with prompt-level disagreement and higher error.<n>A training-free mean ensemble improves NLL on all 14 dataset-model evaluation pairs.<n>We frame this as a reliability stress test for zero-shot VLM first-token safety scores.
  arXiv  Detail & Related papers  (2026-05-01T01:06:30Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• VAL-Bench: Measuring Value Alignment in Language Models [10.745372809345412]
  Large language models (LLMs) are increasingly used for tasks where outputs shape human decisions.<n>Existing benchmarks mostly track refusals or predefined safety violations but do not reveal whether a model upholds a coherent value system.<n>We introduce the Value ALignment Benchmark (VAL-Bench), which evaluates whether models maintain a stable value stance across paired that frame opposing sides of public debates.
  arXiv  Detail & Related papers  (2025-10-06T23:55:48Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity [21.192000569821943]
  We argue that without tight objectives and verifiable constructions, benchmark rankings can produce high-confidence rankings that are in fact largely noise.<n>We show that the ELO-style aggregation used by Arena-Hard Auto collapses and masks genuine ranking uncertainty.<n>Our results highlight design failures that undermine validity and offer actionable principles for building better-scoped, reliability-aware benchmarks.
  arXiv  Detail & Related papers  (2025-09-24T16:26:47Z)
• Uncertainty in Language Models: Assessment through Rank-Calibration [65.10149293133846]
  Language Models (LMs) have shown promising performance in natural language generation. It is crucial to correctly quantify their uncertainty in responding to given inputs. We develop a novel and practical framework, termed $Rank$-$Calibration$, to assess uncertainty and confidence measures for LMs.
  arXiv  Detail & Related papers  (2024-04-04T02:31:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.