Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models
- URL: http://arxiv.org/abs/2604.10949v1
- Date: Mon, 13 Apr 2026 03:46:45 GMT
- Title: Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models
- Abstract summary: Unified models (UMMs) were designed to combine the reasoning ability of large language models (LLMs) with the generation capability of vision models.<n>UMMs fail to transfer LLM-like reasoning to image synthesis and exhibit divergent response behaviors.<n>We propose an information-theoretic probing framework that jointly analyzes how UMMs encode inputs and generate outputs.
- Score: 34.752290343546754
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Unified multimodal models (UMMs) were designed to combine the reasoning ability of large language models (LLMs) with the generation capability of vision models. In practice, however, this synergy remains elusive: UMMs fail to transfer LLM-like reasoning to image synthesis and exhibit divergent response behaviors. We term this phenomenon pseudo-unification. Diagnosing its internal causes is important, but existing probing methods either lack model-internal insight or ignore prompt-response dependencies. To address these limitations, we propose an information-theoretic probing framework that jointly analyzes how UMMs encode inputs and generate outputs. Applied to ten representative UMMs, our framework reveals that pseudo-unification stems from a dual divergence: (i) Modality-Asymmetric Encoding, where vision and language follow different entropy trajectories, and (ii) Pattern-Split Response, where text generation exhibits high-entropy creativity while image synthesis enforces low-entropy fidelity. Only models that unify both sides (e.g., via contextual prediction) achieve more genuine unification, enabling stronger reasoning-based text-to-image generation even with fewer parameters. Our work provides the first model-internal probing of unification, demonstrating that real multimodal synergy requires consistency in information flow, not just shared parameters.
Related papers
- Free Lunch for Unified Multimodal Models: Enhancing Generation via Reflective Rectification with Inherent Understanding [20.397510070808238]
Unified Multimodal Models (UMMs) aim to integrate visual understanding and generation within a single structure.<n>UMMs exhibit a notable capability mismatch, where their understanding capability significantly outperforms their generation.<n>We propose UniRect-CoT, a training-free unified rectification chain-of-thought framework.
arXiv Detail & Related papers (2026-04-15T06:41:56Z) - UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations [45.861324782424326]
Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap.<n>We introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation.
arXiv Detail & Related papers (2026-03-11T12:14:26Z) - UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? [50.92401586025528]
Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear.<n>We introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks.
arXiv Detail & Related papers (2026-03-03T18:36:16Z) - UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
arXiv Detail & Related papers (2026-02-12T18:59:49Z) - Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
arXiv Detail & Related papers (2026-01-07T16:39:34Z) - ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation [79.17352367219736]
ROVER tests the use of one modality to guide, verify, or refine outputs in the other.<n>ROVER is a human-annotated benchmark that explicitly targets reciprocal cross-modal reasoning.
arXiv Detail & Related papers (2025-11-03T02:27:46Z) - Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark [69.8473923357969]
Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration.<n>We present Uni-MMMU, a comprehensive benchmark that unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains.
arXiv Detail & Related papers (2025-10-15T17:10:35Z) - Cantor: Inspiring Multimodal Chain-of-Thought of MLLM [83.6663322930814]
We argue that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks.
We propose an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture.
Our experiments demonstrate the efficacy of the proposed framework, showing significant improvements in multimodal CoT performance.
arXiv Detail & Related papers (2024-04-24T17:59:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.