Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

• URL: http://arxiv.org/abs/2604.00547v1
• Date: Wed, 01 Apr 2026 06:45:01 GMT
• Title: Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models
• Abstract summary: Unified Multimodal Large Models (UMLMs) integrate understanding and generation capabilities within a single architecture.<n>Existing safety benchmarks predominantly focus on isolated understanding or generation tasks.<n>We introduce Uni-SafeBench, a comprehensive benchmark featuring a taxonomy of six major safety categories across seven task types.<n>We develop Uni-Judger, a framework that effectively decouples contextual safety from model safety.
• Score: 15.702553632939102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified Multimodal Large Models (UMLMs) integrate understanding and generation capabilities within a single architecture. While this architectural unification, driven by the deep fusion of multimodal features, enhances model performance, it also introduces important yet underexplored safety challenges. Existing safety benchmarks predominantly focus on isolated understanding or generation tasks, failing to evaluate the holistic safety of UMLMs when handling diverse tasks under a unified framework. To address this, we introduce Uni-SafeBench, a comprehensive benchmark featuring a taxonomy of six major safety categories across seven task types. To ensure rigorous assessment, we develop Uni-Judger, a framework that effectively decouples contextual safety from intrinsic safety. Based on comprehensive evaluations across Uni-SafeBench, we uncover that while the unification process enhances model capabilities, it significantly degrades the inherent safety of the underlying LLM. Furthermore, open-source UMLMs exhibit much lower safety performance than multimodal large models specialized for either generation or understanding tasks. We open-source all resources to systematically expose these risks and foster safer AGI development.

Related papers

• Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety [67.86055044301408]
  We present Yuvion LLM, a large language model built for adversarially robust content safety and broader AI safety.<n>Its pipeline combines adversarially aware data construction, knowledge-enhanced continued pretraining, and policy-grounded multi-task safety post-training.<n>Yuvion LLM demonstrates clear advantages on safety-focused benchmarks and particularly strong robustness under adversarial conditions.
  arXiv  Detail & Related papers  (2026-06-26T01:12:02Z)
• Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety [74.88019498257665]
  General-purpose models often struggle to reliably identify and understand real-world multimodal risks.<n>We present Yuvion VL, a family of multimodal large language models purpose-built for content and AI safety.
  arXiv  Detail & Related papers  (2026-06-23T18:00:08Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models [53.41258113970795]
  We investigate whether cross-functionality reciprocity itself constitutes a structural source of vulnerability in Unified Multimodal Models (UMMs)<n>We propose RICE: Reciprocal Interaction-based Cross-functionality Exploitation, a novel attack paradigm that explicitly exploits bidirectional interactions between understanding and generation.
  arXiv  Detail & Related papers  (2026-03-28T16:28:39Z)
• UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models [29.443907124394382]
  We introduce UniSAFE, the first comprehensive benchmark for system-level safety evaluation of Unified Multimodal Models (UMMs)<n>UniSAFE projects common risk scenarios across task-specific I/O configurations, enabling controlled cross-task comparisons of safety failures.<n>Our results reveal critical vulnerabilities across current UMMs, including elevated safety violations in multi-image composition and multi-turn settings.
  arXiv  Detail & Related papers  (2026-03-18T08:30:31Z)
• DeepKnown-Guard: A Proprietary Model-Based Safety Response Framework for AI Agents [12.054307827384415]
  Large Language Models (LLMs) have become increasingly prominent, severely constraining their trustworthy deployment in critical domains.<n>This paper proposes a novel safety response framework designed to safeguard LLMs at both the input and output levels.
  arXiv  Detail & Related papers  (2025-11-05T03:04:35Z)
• SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agents [7.975014390527644]
  Embodied agents powered by large language models (LLMs) inherit advanced planning capabilities; however, their direct interaction with the physical world exposes them to safety vulnerabilities.<n>We present SafeMindBench, a multimodal benchmark with 5,558 samples spanning four task categories (Instr-Risk, Env-Risk, Order-Fix, Req-Align) across high-risk scenarios such as sabotage, harm, privacy, and illegal behavior.<n>We introduce SafeMindAgent, a modular Planner-Executor architecture integrated with three cascaded safety modules, which incorporate safety constraints into the reasoning process.
  arXiv  Detail & Related papers  (2025-09-30T07:24:04Z)
• When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models [50.66979825532277]
  We introduce Safe-Semantics-but-Unsafe-Interpretation (SSUI), the first dataset featuring interpretable reasoning paths tailored for a cross-modal challenge.<n>A novel training framework, Safety-aware Reasoning Path Optimization (SRPO), is also designed based on the SSUI dataset.<n> Experimental results show that our SRPO-trained models achieve state-of-the-art results on key safety benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T15:40:58Z)
• DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models [37.104276926258095]
  Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data.<n>We introduce textbfDREAM (textittextbfDisentangling textbfRisks to textbfEnhance Safety textbfAlignment in textbfMLLMs), a novel approach that enhances safety alignment in MLLMs through supervised fine-tuning and iterative Reinforcement Learning from AI Feedback.
  arXiv  Detail & Related papers  (2025-04-25T03:54:24Z)
• Multimodal Situational Safety [73.63981779844916]
  We present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety.<n>For an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context.<n>We develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs.
  arXiv  Detail & Related papers  (2024-10-08T16:16:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.