Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

• URL: http://arxiv.org/abs/2609.08236v1
• Date: Tue, 08 Sep 2026 04:27:22 GMT
• Title: Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
• Abstract summary: We measure flip rates with paired significance tests and measured noise floors.<n>Findings are precise rather than universal: most judges barely move, but specific judges harbor cheaply exploitable blind spots.<n>We release the dataset, wrappers, code, and per-verdict labels.
• Score: 2.583375572224943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers behind almost every reported jailbreak success rate, defense evaluation, and safety leaderboard. We ask whether these judges grade what a reply contains or how it sounds. We keep a reply's content fixed and add content-invariant style wrappers: fixed strings placed before or after the reply that change only its tone (an educational disclaimer, a fake safety "reasoning" block, a token refusal followed by the unchanged harmful body), or, on harmless refusals, framing that merely sounds dangerous. The body is preserved byte-for-byte, so a faithful judge must return the same verdict, and any flip is an error of the judge, not a change in safety. Over 600 JailbreakBench replies x up to 7 forms x 8 judges, we measure flip rates with paired significance tests and measured noise floors. Findings are precise rather than universal: most judges barely move, but specific judges harbor cheaply exploitable blind spots. A token-refusal wrapper flips 19.9% of GPT-4o-mini's correct "unsafe" verdicts (noise floor 0.5%; 18.2% under majority-of-three re-scoring) yet moves Claude only 0.4%. The deployed Llama Guard 4 is deterministically gamed: an "educational course" framing flips 12.3% of its harmful verdicts to safe. A second deployed guard (gpt-oss-safeguard-20b) is immune, and rewriting only the grading prompt (StrongREJECT-style) cuts the attack tenfold on the identical model -- the vulnerability lives in the judge, not the content. A two-annotator human validation confirms 100% content invariance and 90% of flips as judge errors (kappa 0.95-1.0), and a bootstrap shows the underlying model ranking is already unstable to sampling alone. We release the dataset, wrappers, code, and per-verdict labels.

Related papers

• Jagged Judges: Epistemic Stability Under Silence, Pressure, and Persistence [2.3273914394869393]
  We introduce the emphWiggle Framework, a unified stress test for stability in LLM judges.<n>We use the framework to study 9 frontier models across 14 judging tasks spanning safety, toxicity, AI writing detection, and political-response evaluation.<n>We find that pressure that succeeds in changing a judge's verdict is almost always net-corrupting with respect to ground truth.
  arXiv  Detail & Related papers  (2026-08-12T23:14:05Z)
• More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges [4.167333498582309]
  Training a language model against its own reference-free judgments assumes a model's verdict on a shown answer tracks correctness.<n>We show it fails structurally: conditioned on a candidate, a judge scores plausibility, not correctness, leaving false-positive basins a policy learns to exploit.
  arXiv  Detail & Related papers  (2026-07-07T06:59:30Z)
• How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring [7.789295015218435]
  Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR)<n>We compare the two judge families against human majority votes and then attack them.<n>We recommend that papers report judge precision and recall on a human-labeled slice, report ASR corrected for judge precision, and include an adversarial check of the judge.
  arXiv  Detail & Related papers  (2026-06-24T07:14:17Z)
• LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation [79.03892269184145]
  LivingArena is an automated, contamination-resistant evaluation framework.<n>Models take turns proposing questions, aiming to pose items that opponents cannot answer correctly.<n>Questioners are encouraged to actively identify and exploit opponents' knowledge boundaries.
  arXiv  Detail & Related papers  (2026-06-19T06:20:58Z)
• Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges [65.92623263645139]
  We introduce BITE, a black-box adversarial framework that learns semantics-preserving edits to mislead an LLM judge.<n>BITE achieves an attack success rate exceeding 65% and raises scores by 1-2 points on a 9-point scale.<n>Our findings expose a fundamental weakness in the LLM-as-a-judge paradigm and motivate robust, attack-aware evaluation.
  arXiv  Detail & Related papers  (2026-05-24T05:24:09Z)
• How Sensitive Are Safety Benchmarks to Judge Configuration Choices? [5.94231111588812]
  Safety benchmarks such as HarmBench rely on judges to classify model responses as harmful or safe.<n>We show that the combination of judge model and judge prompt is typically treated as a fixed implementation detail.<n>We find that prompt wording alone, holding the judge model fixed, shifts measured harmful-response rates by up to 24.2 percentage points.
  arXiv  Detail & Related papers  (2026-04-27T05:59:59Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Evaluative Fingerprints: Stable and Systematic Differences in LLM Evaluator Behavior [0.0]
  We find that judges are consistent, but not with each other; they are consistent with themselves.<n>Across 3,240 evaluations, inter-judge agreement is near-zero.<n>Averaging judges' scores produces a synthetic verdict that corresponds to no judge's actual values.
  arXiv  Detail & Related papers  (2026-01-08T17:02:22Z)
• A Granular Study of Safety Pretraining under Model Abliteration [64.24346997570275]
  We study model abliteration, a lightweight projection technique designed to remove refusal-sensitive directions.<n>We issue 100 prompts with balanced harmful and harmless cases, classify responses as **Refusal** or **Non-Refusal** using multiple judges, and validate judge fidelity.<n>Our study produces a checkpoint-level characterization of which data-centric safety components remain robust under abliteration.
  arXiv  Detail & Related papers  (2025-10-03T07:01:45Z)
• JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring [45.76641811031552]
  We introduce JADES (Jailbreak Assessment via Decompositional Scoring), a universal jailbreak evaluation framework.<n>Its key mechanism is to automatically decompose an input harmful question into a set of weighted sub-questions, score each sub-answer, and weight-aggregate the sub-scores into a final decision.<n>We validate JADES on JailbreakQR, a newly introduced benchmark proposed in this work consisting of 400 pairs of jailbreak prompts and responses, each meticulously annotated by humans.
  arXiv  Detail & Related papers  (2025-08-28T14:40:27Z)
• JudgeLM: Fine-tuned Large Language Models are Scalable Judges [48.053949045598344]
  evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively.<n>We propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks.<n>We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors.
  arXiv  Detail & Related papers  (2023-10-26T17:48:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.