Validity-Aware Jailbreak Evaluation for Large Language Models

• URL: http://arxiv.org/abs/2609.00498v1
• Date: Mon, 31 Aug 2026 23:57:53 GMT
• Title: Validity-Aware Jailbreak Evaluation for Large Language Models
• Abstract summary: We propose a verification-centric jailbreak evaluation framework that decomposes responses into ordered steps and evaluates both validity and correctness.<n>SEAV combines semantic interpretation with retrieval-grounded verification using external knowledge sources.<n>We show that enforcing correctness substantially reshapes measured robustness.
• Score: 13.23443638650052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausibility rather than correctness. We identify a key limitation in existing evaluations: many jailbreak intents depend on instructional validity rather than epistemic factuality, allowing realistic-looking responses to be labeled successful despite being factually or procedurally incorrect. To address this gap, we propose Sequential Epistemic and Action-Level Validation (SEAV), a verification-centric jailbreak evaluation framework that decomposes responses into ordered steps and evaluates both validity and correctness. SEAV combines LLM-as-a-judge mechanisms for semantic interpretation with retrieval-grounded verification using external knowledge sources, assessing whether generated content is factually correct, structurally consistent, and operationally capable of advancing harmful objectives. Empirically, SEAV cuts the false-positive rate on SD-A (a curated strategic-dishonesty diagnostic) by 14.9\,pp vs. the strongest baseline, and reclassifies 22.1\%--51.0\% of sampled prior-labeled successes as invalid across three of four public benchmarks. Together, these results show that enforcing correctness substantially reshapes measured robustness: many previously labeled jailbreak successes are reclassified as invalid, and results are stable across the tested search backends and evaluator models. Code and data are available at https://github.com/Ardor-Wu/SEAV.

Related papers

• Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis [31.25049774595995]
  We study whether refusal behavior remains intact (Aligned) or has been refusalablated (Abliterated) within same-lineage models.<n>We evaluate Aligned and Abliterated states on vulnerability detection, CWE attribution, vulnerableline localization, root-cause localization, and executable patch validation.
  arXiv  Detail & Related papers  (2026-07-07T05:04:48Z)
• Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics [50.36375380196006]
  Jailbreak prompts can bypass alignment guardrails in large language models.<n>Prior detection approaches largely rely on a fixed metric space.<n>We show this assumption breaks under (i) pseudo-malicious prompts that are benign by intent but contain safety-related keywords.<n>We present Manifold Trajectory Kinetics (MTK), which treats an LLM as a kinetic system transforming inputs into outputs.
  arXiv  Detail & Related papers  (2026-06-05T14:49:26Z)
• Efficient Self-Evaluation for Diffusion Language Models via Sequence Regeneration [48.19579266939883]
  Diffusion large language models (dLLMs) have attracted significant attention for their ability to enhance diversity, controllability, and parallelism.<n>We propose DiSE, a simple yet effective self-evaluation confidence quantification method for dLLMs.
  arXiv  Detail & Related papers  (2026-03-03T08:58:20Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• The Semantic Trap: Do Fine-tuned LLMs Learn Vulnerability Root Cause or Just Functional Pattern? [14.472036099680961]
  We propose TrapEval, a comprehensive evaluation framework designed to disentangle vulnerability root cause from functional pattern.<n>We fine-tune five representative state-of-the-art LLMs across three model families and evaluate them under cross-dataset testing, semantic-preservings, and varying degrees of semantic gap measured by CodeBLEU.<n>Our findings serve as a wake-up call: high benchmark scores on traditional datasets may be illusory, masking the model's inability to understand the true causal logic of vulnerabilities.
  arXiv  Detail & Related papers  (2026-01-30T07:19:17Z)
• NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks [8.416892421891761]
  Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines.<n>This work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns.<n>A novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors.<n>It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection.
  arXiv  Detail & Related papers  (2025-11-14T14:43:54Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning [64.32925552574115]
  ARMOR is a large language model that analyzes jailbreak strategies and extracts the core intent.<n> ARMOR achieves state-of-the-art safety performance, with an average harmful rate of 0.002 and an attack success rate of 0.06 against advanced optimization-based jailbreaks.
  arXiv  Detail & Related papers  (2025-07-14T09:05:54Z)
• MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning [15.009686577209278]
  We propose an effective method for jailbreaking large language models via Iterative Semantic Tuning, named MIST.<n>MIST enables attackers to iteratively refine prompts that preserve the original semantic intent while inducing harmful content.<n>Results show that MIST achieves competitive attack success rate, relatively low query count, and fair transferability.
  arXiv  Detail & Related papers  (2025-06-20T07:16:47Z)
• Everything You Wanted to Know About LLM-based Vulnerability Detection But Were Afraid to Ask [30.819697001992154]
  Large Language Models are a promising tool for automated vulnerability detection.<n>Despite widespread adoption, a critical question remains: Are LLMs truly effective at detecting real-world vulnerabilities?<n>This paper challenges three widely held community beliefs: that LLMs are (i) unreliable, (ii) insensitive to code patches, and (iii) performance-plateaued across model scales.
  arXiv  Detail & Related papers  (2025-04-18T05:32:47Z)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective [57.57786477441956]
  We propose an adaptive and semantic optimization problem over the population of responses.<n>Our objective doubles the attack success rate (ASR) on Llama3 and increases the ASR from 2% to 50% with circuit breaker defense.
  arXiv  Detail & Related papers  (2025-02-24T15:34:48Z)
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models [123.66104233291065]
  Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. JailbreakBench is an open-sourced benchmark with the following components.
  arXiv  Detail & Related papers  (2024-03-28T02:44:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.