Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards

• URL: http://arxiv.org/abs/2608.07535v1
• Date: Mon, 27 Jul 2026 20:18:25 GMT
• Title: Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
• Abstract summary: Multi-modal large language models (MLLMs) integrate heterogeneous modalities through modality alignment and fusion.<n>Increased model complexity and cross-modal interactions give rise to novel threats, including compromised modality integration, modality misalignment, and fused safety risks.
• Score: 39.496369275408725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal large language models (MLLMs) integrate heterogeneous modalities through modality alignment and fusion, enabling stronger understanding and reasoning. However, this architectural shift reshapes the safety landscape of machine learning. Increased model complexity and cross-modal interactions give rise to novel threats, including compromised modality integration, modality misalignment, and fused safety risks, reflecting shifts in threat modeling beyond uni-modal assumptions. These shifts, in turn, impose new constraints on safety solutions not captured by existing frameworks rooted in uni-modal learning. Motivated by these challenges, this survey provides a systematic analysis of the evolving safety landscape of MLLMs. We first propose a multimodal grounded taxonomy of safety threats and analyze shifts in threat models, covering adversarial attacks, data poisoning, jailbreaks, and hallucinations. We then summarize updated safety assumptions and organize recent advances in MLLM safety strategies accordingly. Finally, we discuss open challenges and future directions to inform the development of more principled and scalable safety mechanisms for multimodal systems.

Related papers

• Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey [0.21847754147782886]
  Multimodal large language models (MLLMs) integrate information from multiple modalities such as text, images, audio, and video, enabling complex capabilities such as visual question answering and audio translation.<n>This survey provides a comprehensive and systematic analysis of adversarial threats to MLLMs.<n>We introduce a taxonomy that organizes adversarial attacks according to attacker objectives, unifying diverse attack surfaces across modalities and deployment settings.<n>We also present a vulnerability-centric analysis that links integrity attacks, safety and jailbreak failures, control and instruction hijacking, and training-time poisoning to shared architectural and representational weaknesses in multimodal systems.
  arXiv  Detail & Related papers  (2026-03-30T00:16:31Z)
• Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models [53.41258113970795]
  We investigate whether cross-functionality reciprocity itself constitutes a structural source of vulnerability in Unified Multimodal Models (UMMs)<n>We propose RICE: Reciprocal Interaction-based Cross-functionality Exploitation, a novel attack paradigm that explicitly exploits bidirectional interactions between understanding and generation.
  arXiv  Detail & Related papers  (2026-03-28T16:28:39Z)
• Automating Steering for Safe Multimodal Large Language Models [58.36932318051907]
  We introduce a modular and adaptive inference-time intervention technology, AutoSteer, without requiring any fine-tuning of the underlying model.<n>AutoSteer incorporates three core components: (1) a novel Safety Awareness Score (SAS) that automatically identifies the most safety-relevant distinctions among the model's internal layers; (2) an adaptive safety prober trained to estimate the likelihood of toxic outputs from intermediate representations; and (3) a lightweight Refusal Head that selectively intervenes to modulate generation when safety risks are detected.
  arXiv  Detail & Related papers  (2025-07-17T16:04:55Z)
• Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models [83.80177564873094]
  We propose a unified multimodal universal jailbreak attack framework.<n>We evaluate the undesirable context generation of MLLMs like LLaVA, Yi-VL, MiniGPT4, MiniGPT-v2, and InstructBLIP.<n>This study underscores the urgent need for robust safety measures in MLLMs.
  arXiv  Detail & Related papers  (2025-06-02T04:33:56Z)
• DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models [37.104276926258095]
  Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data.<n>We introduce textbfDREAM (textittextbfDisentangling textbfRisks to textbfEnhance Safety textbfAlignment in textbfMLLMs), a novel approach that enhances safety alignment in MLLMs through supervised fine-tuning and iterative Reinforcement Learning from AI Feedback.
  arXiv  Detail & Related papers  (2025-04-25T03:54:24Z)
• Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends [5.233512464561313]
  Traffic safety remains a critical global challenge, with traditional Advanced Driver-Assistance Systems often struggling in dynamic real-world scenarios.<n>This paper reviews the transformative potential of Multimodal Large Language Models (MLLMs) in addressing these limitations.<n>By positioning MLLMs as a cornerstone for next-generation traffic safety systems, this review underscores their potential to revolutionize the field.
  arXiv  Detail & Related papers  (2025-04-21T18:48:35Z)
• SoK: The Security-Safety Continuum of Multimodal Foundation Models through Information Flow and Game-Theoretic Defenses [58.93030774141753]
  Multimodal foundation models (MFMs) integrate diverse data modalities to support complex and wide-ranging tasks.<n>In this paper, we unify the concepts of safety and security in the context of MFMs by identifying critical threats that arise from both model behavior and system-level interactions.
  arXiv  Detail & Related papers  (2024-11-17T23:06:20Z)
• Multimodal Situational Safety [73.63981779844916]
  We present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety.<n>For an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context.<n>We develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs.
  arXiv  Detail & Related papers  (2024-10-08T16:16:07Z)
• Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications [0.0]
  Large Language Models (LLMs) have revolutionized various applications by providing advanced natural language processing capabilities. This paper explores the threat modeling and risk analysis specifically tailored for LLM-powered applications.
  arXiv  Detail & Related papers  (2024-06-16T16:43:58Z)
• Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security [5.077261736366414]
  The pursuit of reliable AI systems like powerful MLLMs has emerged as a pivotal area of contemporary research. In this paper, we endeavor to demostrate the multifaceted risks associated with the incorporation of image modalities into MLLMs.
  arXiv  Detail & Related papers  (2024-04-08T07:54:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.