When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?

• URL: http://arxiv.org/abs/2608.22174v2
• Date: Tue, 25 Aug 2026 04:03:55 GMT
• Title: When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
• Abstract summary: We introduce VGAU-Diag, a fine-grained evaluation framework for vision generation-assisted understanding.<n>We show that generated visual aids help on easier instances but become unreliable as reasoning complexity increases.<n>We also show that effective visual generation should target visual-understanding bottlenecks rather than add more reasoning steps.
• Score: 16.175191237910916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding? Existing evaluations provide mixed evidence, but confound task difficulty, reasoning paradigms, and the closed-loop interaction between generation and understanding. We introduce VGAU-Diag, a fine-grained evaluation framework for vision generation-assisted understanding. It stratifies samples by difficulty, enables unified evaluation of multiple reasoning paradigms, and uses Oracle-Assisted Reference Protocols. Our analysis shows that generated visual aids help on easier instances but become unreliable as reasoning complexity increases. Oracle-assisted diagnosis further reveals that the main bottleneck often lies on the visual-understanding side rather than the visual-generation side, as current UMMs struggle to leverage even faithful visual aids. We also show that effective visual generation should target visual-understanding bottlenecks rather than add more reasoning steps, and identify a three-stage transition from task-irrelevant noise, to misleading plausible guidance, and finally to useful assistance. These findings would be useful to guide the development of better UMMs.

Related papers

• Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models [34.560908829375585]
  We propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step.<n>Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts.<n>We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency.<n>This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.
  arXiv  Detail & Related papers  (2026-05-15T09:48:46Z)
• Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models [98.8608163448532]
  unified multimodal models (UMMs) have made remarkable progress in integrating visual understanding and generation.<n>We propose a token-level intrinsic text-image alignment reward mechanism, GvU, enabling the UMM to act simultaneously as teacher and student.<n>We show that our method substantially boosts UMMs' generation, which in turn strengthens their fine-grained visual understanding.
  arXiv  Detail & Related papers  (2026-03-06T08:56:14Z)
• Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities? [61.533560295383786]
  Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture.<n>We observe that U-MLLMs fail to maintain semantic equivalence when required to render the same results in the image modality.<n>We introduce VGUBench, a framework to decouple reasoning logic from generation fidelity.
  arXiv  Detail & Related papers  (2026-02-27T06:23:56Z)
• MentisOculi: Revealing the Limits of Reasoning with Mental Imagery [63.285794947638614]
  We develop MentisOculi, a suite of multi-step reasoning problems amenable to visual solution.<n> evaluating visual strategies ranging from latent tokens to explicit generated imagery, we find they generally fail to improve performance.<n>Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model reasoning.
  arXiv  Detail & Related papers  (2026-02-02T18:49:06Z)
• Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation [53.18286807225952]
  Unified Multimodal Models (UMMs) integrate both visual understanding and generation within a single framework.<n>We propose UniMRG (Unified Multi-Representation Generation), a simple yet effective architecture-agnostic post-training method.<n>Our method notably enhances fine-grained perception, reduces hallucinations, and improves spatial understanding, while simultaneously boosting generation capabilities.
  arXiv  Detail & Related papers  (2026-01-29T08:42:25Z)
• Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models [51.900488744931785]
  We introduce the Visual Graph Arena (VGA) to evaluate and improve AI systems' capacity for visual abstraction.<n>Humans achieve near-perfect accuracy across tasks, while models totally failed on isomorphism detection and showed limited success in path/cycle tasks.<n>By isolating the challenge of representation-invariant reasoning, the VGA provides a framework to drive progress toward human-like conceptualization in AI visual models.
  arXiv  Detail & Related papers  (2025-06-06T17:06:25Z)
• Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models [26.14137626882127]
  Large Multimodal Models (LMMs) have recently demonstrated remarkable visual understanding performance on both vision-language and vision-centric tasks.<n>We present a unified visual reasoning mechanism that enables LMMs to solve complicated compositional problems.<n>Our trained model, Griffon-R, has the ability of end-to-end automatic understanding, self-thinking, and reasoning answers.
  arXiv  Detail & Related papers  (2025-05-27T05:50:25Z)
• Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning [125.79428219851289]
  Inst-IT is a solution to enhance LMMs in Instance understanding via explicit visual prompt Instruction Tuning.<n>Inst-IT consists of a benchmark to diagnose multimodal instance-level understanding, a large-scale instruction-tuning dataset, and a continuous instruction-tuning training paradigm.
  arXiv  Detail & Related papers  (2024-12-04T18:58:10Z)
• Cantor: Inspiring Multimodal Chain-of-Thought of MLLM [83.6663322930814]
  We argue that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks. We propose an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture. Our experiments demonstrate the efficacy of the proposed framework, showing significant improvements in multimodal CoT performance.
  arXiv  Detail & Related papers  (2024-04-24T17:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.