Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching

• URL: http://arxiv.org/abs/2602.12221v1
• Date: Thu, 12 Feb 2026 17:59:08 GMT
• Title: Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching
• Abstract summary: UniDFlow is a discrete flow-matching framework for multimodal understanding, generation, and editing.<n>UniDFlpw SOTA achieves strong zero-shot generalization to tasks including inpainting, in-context image generation, reference-based editing, and compositional generation.
• Score: 17.68651859356751
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose UniDFlow, a unified discrete flow-matching framework for multimodal understanding, generation, and editing. It decouples understanding and generation via task-specific low-rank adapters, avoiding objective interference and representation entanglement, while a novel reference-based multimodal preference alignment optimizes relative outcomes under identical conditioning, improving faithfulness and controllability without large-scale retraining. UniDFlpw achieves SOTA performance across eight benchmarks and exhibits strong zero-shot generalization to tasks including inpainting, in-context image generation, reference-based editing, and compositional generation, despite no explicit task-specific training.

Related papers

• CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations [4.376075319720958]
  Multi-task learning (MTL) aims to construct a joint model for multiple tasks by sharing a common representation across domains.<n>We propose Causal Orthogonal Representations for Multi-Task Learning (CORE-MTL) to disentangle task-relevant structure from spurious context.<n>CORE-MTL consistently outperforms existing methods on visual multi-task benchmarks in both in-distribution and out-of-distribution settings.
  arXiv  Detail & Related papers  (2026-06-01T13:20:58Z)
• Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning [56.99010101756807]
  We propose Sync-R1, an end-to-end reinforcement learning framework that jointly optimize personalized understanding and generation.<n> Sync-R1 enables personalized comprehension to guide content creation, while the resulting generation quality reciprocally refines understanding.<n>We also introduce Dynamic Group Scaling (DGS), which adaptively filters low-potential trajectories to reduce gradient variance and accelerate convergence.<n> Experimental results demonstrate that Sync-R1 achieves state-of-the-art performance, showcasing superior cross-task reasoning and robust personalization.
  arXiv  Detail & Related papers  (2026-05-11T12:18:26Z)
• UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation [51.41441081823758]
  Unified models capable of interleaved generation have emerged as a promising paradigm.<n>We propose a unified reinforcement learning framework tailored for interleaved generation.<n>Our experiments demonstrate that this unified training recipe significantly enhances image generation quality through reasoning.
  arXiv  Detail & Related papers  (2026-03-24T17:59:17Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Enhancing Semi-Supervised Multi-View Graph Convolutional Networks via Supervised Contrastive Learning and Self-Training [9.300953069946969]
  graph convolutional network (GCN)-based multi-view learning provides a powerful framework for integrating structural information from heterogeneous views.<n>Existing methods often fail to fully exploit the complementary information across views, leading to suboptimal feature representations and limited performance.<n>We propose MV-SupGCN, a semi-supervised GCN model that integrates several complementary components with clear motivations and mutual reinforcement.
  arXiv  Detail & Related papers  (2025-12-15T16:39:23Z)
• UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation [54.38636515750502]
  We propose UniLayDiff: a Unified Diffusion Transformer for content-aware layout generation tasks.<n>We employ Multi-Modal Diffusion Transformer framework to capture the complex interplay between the background image, layout elements, and diverse constraints.<n>Experiments demonstrate that UniLayDiff achieves state-of-the-art performance across from unconditional to various conditional generation tasks.
  arXiv  Detail & Related papers  (2025-12-09T18:38:44Z)
• A Unified Multi-Task Learning Framework for Generative Auto-Bidding with Validation-Aligned Optimization [51.27959658504722]
  Multi-task learning offers a principled framework to train these tasks jointly through shared representations.<n>Existing multi-task optimization strategies are primarily guided by training dynamics and often generalize poorly in volatile bidding environments.<n>We present Validation-Aligned Multi-task Optimization (VAMO), which adaptively assigns task weights based on the alignment between per-task training gradients and a held-out validation gradient.
  arXiv  Detail & Related papers  (2025-10-09T03:59:51Z)
• UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception [54.53657134205492]
  UniAlignment is a unified multimodal generation framework within a single diffusion transformer.<n>It incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model's cross-modal consistency and instruction-following robustness.<n>We present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions.
  arXiv  Detail & Related papers  (2025-09-28T09:11:30Z)
• Unified Autoregressive Visual Generation and Understanding with Continuous Tokens [52.21981295470491]
  We present UniFluid, a unified autoregressive framework for joint visual generation and understanding.<n>Our unified autoregressive architecture processes multimodal image and text inputs, generating discrete tokens for text and continuous tokens for image.<n>We find though there is an inherent trade-off between the image generation and understanding task, a carefully tuned training recipe enables them to improve each other.
  arXiv  Detail & Related papers  (2025-03-17T17:58:30Z)
• Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence [83.15764564701706]
  We propose a novel framework that performs vision-language alignment by integrating Cauchy-Schwarz divergence with mutual information.<n>We find that the CS divergence seamlessly addresses the InfoNCE's alignment-uniformity conflict and serves complementary roles with InfoNCE.<n> Experiments on text-to-image generation and cross-modality retrieval tasks demonstrate the effectiveness of our method on vision-language alignment.
  arXiv  Detail & Related papers  (2025-02-24T10:29:15Z)
• Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionality [69.76121008898677]
  Fine-grained Selective Calibrated CLIP integrates local hard negative loss and selective calibrated regularization. Our evaluations show that FSC-CLIP not only achieves compositionality on par with state-of-the-art models but also retains strong multi-modal capabilities.
  arXiv  Detail & Related papers  (2024-10-07T17:16:20Z)
• Generalized Adversarially Learned Inference [42.40405470084505]
  We develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs. We incorporate multiple layers of feedback on reconstructions, self-supervision, and other forms of supervision based on prior or learned knowledge about the desired solutions.
  arXiv  Detail & Related papers  (2020-06-15T02:18:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.