LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation

• URL: http://arxiv.org/abs/2603.27693v1
• Date: Sun, 29 Mar 2026 13:38:21 GMT
• Title: LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation
• Abstract summary: Unified multimodal pretraining has emerged as a promising paradigm for jointly modeling language and vision within a single foundation model.<n>Existing approaches largely rely on implicit or indirect alignment signals and remain suboptimal for simultaneously supporting multimodal understanding and generation.<n>We propose LVRPO, a language-visual reinforcement-based preference optimization framework that explicitly aligns language and visual representations.
• Score: 51.071351994330605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multimodal pretraining has emerged as a promising paradigm for jointly modeling language and vision within a single foundation model. However, existing approaches largely rely on implicit or indirect alignment signals and remain suboptimal for simultaneously supporting multimodal understanding and generation, particularly in settings that require fine-grained language-visual reasoning and controllable generation. In this work, we propose LVRPO, a language-visual reinforcement-based preference optimization framework that explicitly aligns language and visual representations using Group Relative Policy Optimization (GRPO). Instead of introducing additional alignment losses at the representation level, LVRPO directly optimizes multimodal model behaviors through preference-driven reinforcement signals, encouraging consistent and semantically grounded interactions between language and vision across both understanding and generation tasks. This formulation enables effective alignment without requiring auxiliary encoders or handcrafted cross-modal objectives, and naturally extends to diverse multimodal capabilities. Empirically, LVRPO consistently outperforms strong unified-pretraining baselines on a broad suite of benchmarks spanning multimodal understanding, generation, and reasoning.

Related papers

• Multi-Branch Policy Optimization for Multimodal Large Language Models [21.856599797665833]
  Group-based reinforcement learning methods for multimodal large language models rely on trajectory-level credit assignment.<n> multimodal reasoning involves substantially higher perceptual uncertainty than text-only settings.<n>We propose Multi-Branch Policy Optimization (MBPO), a tree-based framework that constructs reasoning trees at vision-language decision boundaries.
  arXiv  Detail & Related papers  (2026-08-05T04:00:09Z)
• Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization [35.14373974143734]
  We propose a reinforcement learning-based post-training strategy to unlock this capability in existing unified models.<n>Our approach jointly models text and image generation within a single decoding trajectory and optimize it with our novel hybrid rewards.<n>Experiments on MMIE and InterleavedBench demonstrate that our approach significantly enhances the quality and coherence of multimodal interleaved generation.
  arXiv  Detail & Related papers  (2026-03-10T11:49:20Z)
• Multi-Paradigm Collaborative Adversarial Attack Against Multi-Modal Large Language Models [67.45032003041399]
  We propose a novel Multi-Paradigm Collaborative Attack (MPCAttack) framework to boost the transferability of adversarial examples against MLLMs.<n>MPCO adaptively balances the importance of different paradigm representations and guides the global optimisation.<n>Our solution consistently outperforms state-of-the-art methods in both targeted and untargeted attacks on open-source and closed-source MLLMs.
  arXiv  Detail & Related papers  (2026-03-05T06:01:26Z)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining [125.34714978184638]
  We provide empirical clarity through controlled, from-scratch pretraining experiments.<n>We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision.<n>We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language.
  arXiv  Detail & Related papers  (2026-03-03T18:58:00Z)
• HMVLA: Hyperbolic Multimodal Fusion for Vision-Language-Action Models [4.59200581394731]
  HMVLA exploits inherent hierarchical structures in vision and language for comprehensive semantic alignment.<n>Our HMVLA embeds multimodal features in hyperbolic space, enabling more effective modeling of the hierarchical relationships present in image text data.
  arXiv  Detail & Related papers  (2026-01-28T07:50:30Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• Scaling Language-Centric Omnimodal Representation Learning [26.999264997449586]
  multimodal embedding approaches leveraging large language models (MLLMs) fine-tuned with contrastive learning (CL) have shown promising results.<n>This work argues that a crucial advantage of MLLM-based approaches stems from implicit cross-modal alignment achieved during generative pretraining.<n>We propose a Language-Centric Omnimodal Embedding framework, termed LCO-Emb.
  arXiv  Detail & Related papers  (2025-10-13T17:53:52Z)
• UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception [54.53657134205492]
  UniAlignment is a unified multimodal generation framework within a single diffusion transformer.<n>It incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model's cross-modal consistency and instruction-following robustness.<n>We present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions.
  arXiv  Detail & Related papers  (2025-09-28T09:11:30Z)
• Improving Multilingual Language Models by Aligning Representations through Steering [10.159957091670883]
  This paper investigates how Large Language Models (LLMs) represent non-English tokens.<n>We propose a lightweight intervention method using representation steering, where a learned vector is added to the residual stream at a single model layer to enhance multilingual performance.
  arXiv  Detail & Related papers  (2025-05-19T00:14:43Z)
• Relation-R1: Progressively Cognitive Chain-of-Thought Guided Reinforcement Learning for Unified Relation Comprehension [31.952192907460713]
  Relation-R1 is the textitfirst unified relation comprehension framework.<n>It integrates cognitive chain-of-thought (CoT)-guided supervised fine-tuning (SFT) and group relative policy optimization ( GRPO)<n>Experiments on widely-used PSG and SWiG datasets demonstrate that Relation-R1 achieves state-of-the-art performance in both binary and textitN-ary relation understanding.
  arXiv  Detail & Related papers  (2025-04-20T14:50:49Z)
• Unified Generative and Discriminative Training for Multi-modal Large Language Models [88.84491005030316]
  Generative training has enabled Vision-Language Models (VLMs) to tackle various complex tasks. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval. This paper proposes a unified approach that integrates the strengths of both paradigms.
  arXiv  Detail & Related papers  (2024-11-01T01:51:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.