Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks

• URL: http://arxiv.org/abs/2607.26574v1
• Date: Wed, 29 Jul 2026 07:53:35 GMT
• Title: Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks
• Abstract summary: We build a guard-agnostic recover-and-decode amplifier for vision-language models.<n>We evaluate it against an ensemble of eleven attacks, scoring a behavior as broken if any succeeds.<n>This exposes our central finding: an empirical safety-utility ceiling for the non-iterative recovery defenses we evaluate.
• Score: 7.287314790466206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a rare language, code, or an image of text slips past a guard that would block it in plain language -- the decode gap. The natural fix is a guard-agnostic recover-and-decode amplifier that transcribes image content and restates encoded text into its plain payload before the guard, so any off-the-shelf classifier can screen the true request. We build this amplifier and evaluate it against the attacker's best case: an ensemble of eleven attacks, scoring a behavior as broken if any succeeds (best-of-suite, following AutoAttack) -- rarely reported for jailbreak defenses, yet ~3.5x the per-attack mean. This exposes our central finding: an empirical safety-utility ceiling for the non-iterative recovery defenses we evaluate, across five guards and two target VLMs. The amplifier only partly closes the gap -- the undefended ensemble breaks 89-91% of behaviors, and the best guard-plus-amplifier still leaves 63-65% -- and its gain over the guard alone is significant in only four of ten guard-target pairs. It is guard-agnostic at the interface, but not uniformly so in effect. A modular reguard layer closes much of the residual, yet drives benign over-refusal to 81-92% for well-calibrated guards; the one laxer guard that stays usable never reaches deployable safety (48% ensemble ASR). No configuration we evaluate reaches both low attack-success and low over-refusal, for the pipeline we study and for representation-shifting attacks -- encodings and cross-modal renders that leave a legible payload, not pixel- or embedding-space attacks. We contribute the amplifier, an ensemble evaluation that makes the trade-off visible, and a map of where recovery-based VLM defense works and where it does not.

Related papers

• AlcaTRAz - Anchored Tree-Rule Defense Against Jailbreaks [0.08699280339422538]
  Large language models (LLMs) are vulnerable to jailbreak attacks that bypass safety alignment through carefully crafted prompts.<n>We propose AlcaTRAz, a prompt-level defense based on rule trees that operates exclusively on the input text.<n>We evaluate the proposed method across 33 open-weight models, 22 jailbreak attack types, and a benchmark of short, single-turn benign questions.
  arXiv  Detail & Related papers  (2026-09-03T11:30:08Z)
• Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails [24.794304039477495]
  We show that the guardrail's actual share of the safety credited to it runs from none of it to essentially all of it.<n>The split is recoverable at no extra cost, because a guard block replaces the model's response.
  arXiv  Detail & Related papers  (2026-08-09T11:19:30Z)
• DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models [8.719976368689826]
  Cloud-based language model services process prompts containing sensitive information.<n>Obfuscation-based defenses transform prompt representations before transmission, offering a lightweight alternative to cryptographic solutions.<n>We show these defenses provide far less protection than previously believed.<n>We present DeepInvert, a semi-supervised embedding inversion attack that recovers original tokens from obfuscated representations with higher accuracy than prior methods.
  arXiv  Detail & Related papers  (2026-08-05T06:02:51Z)
• When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation [9.68749971398651]
  A vision-language AI assistant returns its answer as a stream of generated tokens.<n>A safety guard that watches that answer has to keep up with the stream and stop a harmful reply before a user reads it.<n>Recent vision-language guardrails instead generate a chain of thought before they issue a verdict.
  arXiv  Detail & Related papers  (2026-07-23T15:02:24Z)
• SentGuard: Sentence-Level Streaming Guardrails for Large Language Models [40.00517152303363]
  We propose SentGuard, a sentence-level streaming guardrail that operates in parallel with generation.<n>A benchmark with structured per-sentence annotations captures the evolution of safety risks across both reasoning and response segments.<n>Experiments show that SentGuard outperforms existing baselines, detecting 90.5% of unsafe cases within two sentences while maintaining a low streaming false-positive rate of 7.41%.
  arXiv  Detail & Related papers  (2026-06-01T10:30:08Z)
• TrapSuffix: Proactive Defense Against Adversarial Suffixes in Jailbreaking [52.72486831074384]
  Suffix-based jailbreak attacks append an adversarial suffix, i.e., a short token sequence, to steer aligned LLMs into unsafe outputs.<n>We propose TrapSuffix, a lightweight fine-tuning approach that injects trap-aligned behaviors into the base model without changing the inference pipeline.<n>Across diverse suffix-based jailbreak settings, TrapSuffix reduces the average attack success rate to below 0.01 percent and achieves an average tracing success rate of 87.9 percent.
  arXiv  Detail & Related papers  (2026-02-06T11:43:56Z)
• TRYLOCK: Defense-in-Depth Against LLM Jailbreaks via Layered Preference and Representation Engineering [0.0]
  We present TRYLOCK, the first defense-in-depth architecture that combines four mechanisms across the inference stack.<n>On Mistral-7B-Instruct evaluated against a 249-prompt attack set, TRYLOCK achieves 88.0% relative ASR reduction.
  arXiv  Detail & Related papers  (2026-01-06T03:02:20Z)
• Multi-Faceted Attack: Exposing Cross-Model Vulnerabilities in Defense-Equipped Vision-Language Models [54.61181161508336]
  We introduce Multi-Faceted Attack (MFA), a framework that exposes general safety vulnerabilities in leading defense-equipped Vision-Language Models (VLMs)<n>The core component of MFA is the Attention-Transfer Attack (ATA), which hides harmful instructions inside a meta task with competing objectives.<n>MFA achieves a 58.5% success rate and consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2025-11-20T07:12:54Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• Bypassing Prompt Guards in Production with Controlled-Release Prompting [11.65770031195044]
  We introduce a new attack that circumvents prompt guards, highlighting their limitations.<n>Our method consistently jailbreaks production models while maintaining response quality.<n>This reveals an attack surface inherent to lightweight prompt guards in modern LLM architectures.
  arXiv  Detail & Related papers  (2025-10-02T00:04:21Z)
• The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense [56.32083100401117]
  The vulnerability of Vision Large Language Models (VLLMs) to jailbreak attacks appears as no surprise.<n>Recent defense mechanisms against these attacks have reached near-saturation performance on benchmark evaluations.
  arXiv  Detail & Related papers  (2024-11-13T07:57:19Z)
• Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing [107.97160023681184]
  Aligned large language models (LLMs) are vulnerable to jailbreaking attacks. We propose SEMANTICSMOOTH, a smoothing-based defense that aggregates predictions of semantically transformed copies of a given input prompt.
  arXiv  Detail & Related papers  (2024-02-25T20:36:03Z)
• TextGuard: Provable Defense against Backdoor Attacks on Text Classification [83.94014844485291]
  We propose TextGuard, the first provable defense against backdoor attacks on text classification. In particular, TextGuard divides the (backdoored) training data into sub-training sets, achieved by splitting each training sentence into sub-sentences. In our evaluation, we demonstrate the effectiveness of TextGuard on three benchmark text classification tasks.
  arXiv  Detail & Related papers  (2023-11-19T04:42:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.