On the Adversarial Robustness of Multimodal LLM Judges

• URL: http://arxiv.org/abs/2606.15608v1
• Date: Sun, 14 Jun 2026 05:30:20 GMT
• Title: On the Adversarial Robustness of Multimodal LLM Judges
• Abstract summary: We introduce RobustMLLMJudge, the first framework for evaluating the adversarial robustness of general-purpose MLLMs when functioning as judges.<n>Using RobustMLLMJudge, we reveal that different MLLM judges are highly vulnerable to score-inflating adversarial attacks.<n>We propose MGSIA, a novel method that bypasses these constraints to enable more effective and transferable attacks on MLLM judges.
• Score: 49.80009278197918
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) are increasingly used as automated judges, e.g., for image quality and safety assessment. However, their adversarial robustness remains largely unexplored, threatening the fairness and reliability of automated judging. To bridge this gap, we introduce RobustMLLMJudge, the first general framework for evaluating the adversarial robustness of general-purpose MLLMs when functioning as judges. It covers diverse attacks against popular judge approaches across quality and safety evaluation scenarios. Using RobustMLLMJudge, we reveal that i) different MLLM judges are highly vulnerable to score-inflating adversarial attacks; and ii) although effective, these attack methods face a critical challenge due to unique constraints in the evaluation protocols of MLLM judges. We further propose MGSIA, namely Manifold-Guided Semantic Induction Attack, a novel method that bypasses these constraints to enable more effective and transferable attacks on MLLM judges. The core idea of MGSIA is to combine affirmative semantic induction with high-score manifold alignment: it maximizes the probability that judges yield affirmative responses (e.g., "Yes") to binary semantic queries, while regularizing adversarial representations toward high-score centers estimated from proxy protocols. Together, these objectives yield transferable score-inflating perturbations. Extensive experiments demonstrate the superiority and generalizability of MGSIA in deceiving advanced MLLM judges under different evaluation scenarios, highlighting the need for robust MLLM judges. Code and data will be made available at https://github.com/mala-lab/RobustMLLMJudge.

Related papers

• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Efficient LLM Safety Evaluation through Multi-Agent Debate [18.818180932660294]
  We propose a cost-efficient multi-agent judging framework that employs Small Language Models (SLMs) through structured debates among critic, defender, and judge agents.<n>To rigorously assess safety judgments, we construct HAJailBench, a large-scale human-annotated jailbreak benchmark.<n>Our framework achieves agreement comparable to GPT-4o judges on HAJailBench while substantially reducing inference cost.
  arXiv  Detail & Related papers  (2025-11-09T14:06:55Z)
• CompassJudger-2: Towards Generalist Judge Model via Verifiable Rewards [72.44810390478229]
  CompassJudger-2 is a novel generalist judge model that overcomes limitations via a task-driven, multi-domain data curation strategy.<n> CompassJudger-2 achieves superior results across multiple judge and reward benchmarks.
  arXiv  Detail & Related papers  (2025-07-12T01:34:24Z)
• LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge [44.6358611761225]
  Large Language Models (LLMs) have demonstrated remarkable intelligence across various tasks.<n>These systems are susceptible to adversarial attacks that can manipulate evaluation outcomes.<n>Existing evaluation methods adopted by LLM-based judges are often piecemeal and lack a unified framework for comprehensive assessment.
  arXiv  Detail & Related papers  (2025-06-11T06:48:57Z)
• Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks [0.0]
  Large Language Models (LLMs) are increasingly employed as evaluators (LLM-as-a-Judge) for assessing the quality of machine-generated text.<n>This paper investigates the vulnerability of LLM-as-a-Judge architectures to prompt-injection attacks.
  arXiv  Detail & Related papers  (2025-05-19T16:51:12Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges [3.168632659778101]
  We highlight two critical challenges that are typically overlooked: (i) evaluations in the wild where factors like prompt sensitivity and distribution shifts can affect performance and (ii) adversarial attacks that target the judge.<n>We show that small changes such as the style of the model output can lead to jumps of up to 0.24 in the false negative rate on the same dataset, whereas adversarial attacks on the model generation can fool some judges into misclassifying 100% of harmful generations as safe ones.
  arXiv  Detail & Related papers  (2025-03-06T14:24:12Z)
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment [8.948475969696075]
  Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations such as assessing written exams and benchmarking systems. We show that short universal adversarial phrases can be deceived to judge LLMs to predict inflated scores. It is found that judge-LLMs are significantly more susceptible to these adversarial attacks when used for absolute scoring.
  arXiv  Detail & Related papers  (2024-02-21T18:55:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.