DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement

• URL: http://arxiv.org/abs/2605.25328v1
• Date: Mon, 25 May 2026 01:17:32 GMT
• Title: DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
• Abstract summary: Unified Multimodal models (UMMs) have shown impressive performance in both understanding and generation.<n>We identify a fundamental challenge that lies in inductive biases induced by distinct supervision signals.<n>We propose DIVA, a self-improved post-training framework that transforms the representation divergence into interior synergy.
• Score: 42.87874090062771
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lies in inductive biases induced by distinct supervision signals: generation branch prefers high-fidelity, fine-grained representations capable of reconstruction, while the understanding favours semantically discriminative embeddings that remain invariant to task-irrelevant factors. Consequently, optimizing these complementary but non-equivalent objectives within a monolithic backbone leads to mutual impairment instead of enhancement. In this paper, we first analyze the root cause of this interference in unified backbones and reveal a complementary structure in their internal representations. Motivated by the observation, we propose DIVA, a self-improved post-training framework that transforms the representation divergence into interior synergy. By explicitly factorizing the visual representation into shared and unique components based on two complementary information flow, DIVA enables both the understanding and generation branches to achieve beneficial transferring while preserving the integrity of unique information from cross-flow interference via mutual information estimation. Despite its generality, our method consistently achieves improvements across visual understanding (+7.82%) and generation (+8.46%). The official code is available at: https://github.com/Jayyy-H/DIVA.

Related papers

• RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities [34.35941675436447]
  RePercENT is a self-supervised framework designed to unlock scalable pairwise disentanglement beyond two modalities.<n>Our approach operates directly on pre-extracted embeddings, eliminating the need for extensive joint pre-training.<n>Across diverse modalities and tasks, RePercENT successfully recovers disentangled components while maintaining competitive performance.
  arXiv  Detail & Related papers  (2026-06-03T17:10:39Z)
• UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations [45.861324782424326]
  Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap.<n>We introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation.
  arXiv  Detail & Related papers  (2026-03-11T12:14:26Z)
• Enhancing Semi-Supervised Multi-View Graph Convolutional Networks via Supervised Contrastive Learning and Self-Training [9.300953069946969]
  graph convolutional network (GCN)-based multi-view learning provides a powerful framework for integrating structural information from heterogeneous views.<n>Existing methods often fail to fully exploit the complementary information across views, leading to suboptimal feature representations and limited performance.<n>We propose MV-SupGCN, a semi-supervised GCN model that integrates several complementary components with clear motivations and mutual reinforcement.
  arXiv  Detail & Related papers  (2025-12-15T16:39:23Z)
• Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representational Alignment [13.028121107802127]
  In inverse problems, pretrained generative models are employed as priors.<n>We propose applying representation alignment (REPA) between diffusion or flow-based models and a pretrained self-supervised visual encoder.<n>We show that aligning model representations with approximate target features can substantially enhance reconstruction fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2025-11-21T00:37:04Z)
• Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark [69.8473923357969]
  Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration.<n>We present Uni-MMMU, a comprehensive benchmark that unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains.
  arXiv  Detail & Related papers  (2025-10-15T17:10:35Z)
• Semantic Item Graph Enhancement for Multimodal Recommendation [49.66272783945571]
  Multimodal recommendation systems have attracted increasing attention for their improved performance by leveraging items' multimodal information.<n>Prior methods often build modality-specific item-item semantic graphs from raw modality features.<n>These semantic graphs suffer from semantic deficiencies, including insufficient modeling of collaborative signals among items.
  arXiv  Detail & Related papers  (2025-08-08T09:20:50Z)
• Unity in Diversity: Multi-expert Knowledge Confrontation and Collaboration for Generalizable Vehicle Re-identification [60.20318058777603]
  Generalizable vehicle re-identification (ReID) seeks to develop models that can adapt to unknown target domains without the need for fine-tuning or retraining.<n>Previous works have mainly focused on extracting domain-invariant features by aligning data distributions between source domains.<n>We propose a two-stage Multi-expert Knowledge Confrontation and Collaboration (MiKeCoCo) method to solve this unique problem.
  arXiv  Detail & Related papers  (2024-07-10T04:06:39Z)
• Local Consensus Enhanced Siamese Network with Reciprocal Loss for Two-view Correspondence Learning [35.5851523517487]
  Two-view correspondence learning usually establish an end-to-end network to jointly predict correspondence reliability and relative pose. We propose a Local Feature Consensus (LFC) plugin block to augment the features of existing models. We extend existing models to a Siamese network with a reciprocal loss that exploits the supervision of mutual projection.
  arXiv  Detail & Related papers  (2023-08-06T22:20:09Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• The Transitive Information Theory and its Application to Deep Generative Models [0.0]
  Variational Autoencoder (VAE) could be pushed in two opposite directions. Existing methods narrow the issues to the rate-distortion trade-off between compression and reconstruction. We develop a system that learns a hierarchy of disentangled representation together with a mechanism for recombining the learned representation for generalization.
  arXiv  Detail & Related papers  (2022-03-09T22:35:02Z)
• Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification [208.1227090864602]
  Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing VI-ReID methods tend to learn global representations, which have limited discriminability and weak robustness to noisy images. We propose a novel dynamic dual-attentive aggregation (DDAG) learning method by mining both intra-modality part-level and cross-modality graph-level contextual cues for VI-ReID.
  arXiv  Detail & Related papers  (2020-07-18T03:08:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.