Red Teaming Multimodal Language Models: Evaluating Harm Across Prompt Modalities and Models

• URL: http://arxiv.org/abs/2509.15478v1
• Date: Thu, 18 Sep 2025 22:51:06 GMT
• Title: Red Teaming Multimodal Language Models: Evaluating Harm Across Prompt Modalities and Models
• Authors: Madison Van Doren, Casey Ford, Emily Dix,
• Abstract summary: Multimodal large language models (MLLMs) are increasingly used in real world applications, yet their safety under adversarial conditions remains underexplored.<n>This study evaluates the harmlessness of four leading MLLMs when exposed to adversarial prompts across text-only and multimodal formats.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) are increasingly used in real world applications, yet their safety under adversarial conditions remains underexplored. This study evaluates the harmlessness of four leading MLLMs (GPT-4o, Claude Sonnet 3.5, Pixtral 12B, and Qwen VL Plus) when exposed to adversarial prompts across text-only and multimodal formats. A team of 26 red teamers generated 726 prompts targeting three harm categories: illegal activity, disinformation, and unethical behaviour. These prompts were submitted to each model, and 17 annotators rated 2,904 model outputs for harmfulness using a 5-point scale. Results show significant differences in vulnerability across models and modalities. Pixtral 12B exhibited the highest rate of harmful responses (~62%), while Claude Sonnet 3.5 was the most resistant (~10%). Contrary to expectations, text-only prompts were slightly more effective at bypassing safety mechanisms than multimodal ones. Statistical analysis confirmed that both model type and input modality were significant predictors of harmfulness. These findings underscore the urgent need for robust, multimodal safety benchmarks as MLLMs are deployed more widely.

Related papers

• Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases [0.0]
  Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored.<n>We present a two-phase evaluation of MLLM harmlessness using a fixed benchmark of 726 adversarial prompts authored by 26 professional red teamers.
  arXiv  Detail & Related papers  (2026-02-04T16:42:02Z)
• A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5 [101.4233736714284]
  Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have driven major gains in reasoning, perception, and generation across language and vision.<n>We present an integrated safety evaluation of six frontier models--GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5--assessing each across language, vision-language, and image generation.
  arXiv  Detail & Related papers  (2026-01-15T15:52:52Z)
• Probing Multimodal Large Language Models on Cognitive Biases in Chinese Short-Video Misinformation [34.28647703173823]
  Short-video platforms have become major channels for misinformation, where deceptive claims leverage visual experiments and social cues.<n>We introduce a comprehensive evaluation framework using a high-quality, manually annotated dataset of 200 short videos spanning four health domains.<n>This dataset provides fine-grained annotations for three deceptive patterns, experimental errors, logical fallacies, and fabricated claims.
  arXiv  Detail & Related papers  (2026-01-10T15:43:30Z)
• Adversarial versification in portuguese as a jailbreak operator in LLMs [0.0]
  Recent evidence shows that the versification of prompts constitutes a highly effective adversarial mechanism against aligned LLMs.<n>The absence of evaluations in Portuguese, a language with high morphosyntactic complexity, constitutes a critical gap.
  arXiv  Detail & Related papers  (2025-12-17T11:55:45Z)
• Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models [1.5401871453629499]
  We present evidence that adversarial poetry functions as a universal single-turn jailbreak technique for Large Language Models (LLMs)<n>Across 25 proprietary and open-weight models, curated poetic prompts yielded high attack-success rates (ASR), with some providers exceeding 90%.
  arXiv  Detail & Related papers  (2025-11-19T10:14:08Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability [10.607081850023286]
  We introduce the Input Scrutiny Ability Evaluation Framework (ISEval), which encompasses seven categories of flawed premises and three evaluation metrics.<n>Most models struggle to actively detect flawed textual premises without guidance.<n>These insights underscore the urgent need to enhance LMMs' proactive verification of input validity.
  arXiv  Detail & Related papers  (2025-08-06T02:13:46Z)
• MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks [85.3303135160762]
  MIRAGE is a novel framework that exploits narrative-driven context and role immersion to circumvent safety mechanisms in Multimodal Large Language Models.<n>It achieves state-of-the-art performance, improving attack success rates by up to 17.5% over the best baselines.<n>We demonstrate that role immersion and structured semantic reconstruction can activate inherent model biases, facilitating the model's spontaneous violation of ethical safeguards.
  arXiv  Detail & Related papers  (2025-03-24T20:38:42Z)
• Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models [26.17300490736624]
  Multimodal Large Language Models (MLLMs) are predominantly trained and tested on consistent visual-textual inputs.<n>We propose the Multimodal Inconsistency Reasoning benchmark to assess MLLMs' ability to detect and reason about semantic mismatches.<n>We evaluate six state-of-the-art MLLMs, showing that models with dedicated multimodal reasoning capabilities, such as o1, substantially outperform their counterparts.
  arXiv  Detail & Related papers  (2025-02-22T01:52:37Z)
• MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries? [70.77691645678804]
  Humans are prone to cognitive distortions -- biased thinking patterns that lead to exaggerated responses to specific stimuli. This paper demonstrates that advanced Multimodal Large Language Models (MLLMs) exhibit similar tendencies. We identify three types of stimuli that trigger the oversensitivity of existing MLLMs: Exaggerated Risk, Negated Harm, and Counterintuitive.
  arXiv  Detail & Related papers  (2024-06-22T23:26:07Z)
• Multimodal Large Language Models to Support Real-World Fact-Checking [80.41047725487645]
  Multimodal large language models (MLLMs) carry the potential to support humans in processing vast amounts of information. While MLLMs are already being used as a fact-checking tool, their abilities and limitations in this regard are understudied. We propose a framework for systematically assessing the capacity of current multimodal models to facilitate real-world fact-checking.
  arXiv  Detail & Related papers  (2024-03-06T11:32:41Z)
• How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts [54.07541591018305]
  We present MAD-Bench, a benchmark that contains 1000 test samples divided into 5 categories, such as non-existent objects, count of objects, and spatial relationship. We provide a comprehensive analysis of popular MLLMs, ranging from GPT-4v, Reka, Gemini-Pro, to open-sourced models, such as LLaVA-NeXT and MiniCPM-Llama3. While GPT-4o achieves 82.82% accuracy on MAD-Bench, the accuracy of any other model in our experiments ranges from 9% to 50%.
  arXiv  Detail & Related papers  (2024-02-20T18:31:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.