JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models

• URL: http://arxiv.org/abs/2605.28609v1
• Date: Wed, 27 May 2026 15:24:31 GMT
• Title: JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
• Abstract summary: Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations.<n>Existing adversarial attacks aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment.<n>We propose JECA2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment.
• Score: 1.0998907972211756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. Existing adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes prompt embeddings toward authenticity-affirming semantics under a token-proximity constraint. Experiments on forensic VLM benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.

Related papers

• Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models [58.59008920408855]
  We propose Adversarial Probing for Implicit VulnErabilities (AdvPIE) to expose implicit vulnerabilities without requiring access to the parameters of target models.<n> AdvPIE adopts a policy agent to generate and refine implicit adversarial prompts based on the feedback from a judge agent.<n>Experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities.
  arXiv  Detail & Related papers  (2026-09-05T13:45:09Z)
• GeoThreat: Transferable Targeted Adversarial Attacks on Large Vision-Language Models for Remote Sensing Image Interpretation [25.134254498593137]
  Adversarial attacks against large vision-language models (LVLMs) serve as an effective means of assessing their robustness in cross-modal semantic understanding.<n>We propose GeoThreat, a transferable targeted adversarial attack method against LVLMs for remote sensing image interpretation.
  arXiv  Detail & Related papers  (2026-07-23T08:16:56Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• Causal Bootstrapped Alignment for Unsupervised Video-Based Visible-Infrared Person Re-Identification [52.784239635604735]
  VVI-ReID is a critical technique for all-day surveillance, where temporal information provides additional cues beyond static images.<n>Existing approaches rely heavily on fully supervised learning with expensive cross-modality annotations, limiting scalability.<n>We propose a Causal Bootstrapped Alignment framework that explicitly exploits inherent video priors.
  arXiv  Detail & Related papers  (2026-04-17T02:15:59Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• SCOUT: A Defense Against Data Poisoning Attacks in Fine-Tuned Language Models [11.304852987259041]
  This paper introduces three novel contextually-aware attack scenarios that exploit domain-specific knowledge and semantic plausibility.<n>We present textbfSCOUT (Saliency-based Classification Of Untrusted Tokens), a novel defense framework that identifies backdoor triggers through token-level saliency analysis.
  arXiv  Detail & Related papers  (2025-12-10T17:25:55Z)
• Self-Calibrated Consistency can Fight Back for Adversarial Robustness in Vision-Language Models [31.920092341939593]
  Self-Calibrated Consistency is an effective test-time defense against adversarial attacks.<n> SCC consistently improves the zero-shot robustness of CLIP while maintaining accuracy.<n>These findings highlight the great potential of establishing an adversarially robust paradigm from CLIP.
  arXiv  Detail & Related papers  (2025-10-26T18:37:12Z)
• CeTAD: Towards Certified Toxicity-Aware Distance in Vision Language Models [16.5022773312661]
  We propose a universal certified defence framework to safeguard large vision-language models against jailbreak attacks.<n>First, we proposed a novel distance metric to quantify semantic discrepancies between malicious and intended responses.<n>Then, we devise a regressed certification approach that employs randomized smoothing to provide formal robustness guarantees.
  arXiv  Detail & Related papers  (2025-03-08T17:33:55Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Semantic-Preserving Adversarial Code Comprehension [75.76118224437974]
  We propose Semantic-Preserving Adversarial Code Embeddings (SPACE) to find the worst-case semantic-preserving attacks. Experiments and analysis demonstrate that SPACE can stay robust against state-of-the-art attacks while boosting the performance of PrLMs for code.
  arXiv  Detail & Related papers  (2022-09-12T10:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.