From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space

• URL: http://arxiv.org/abs/2603.12648v1
• Date: Fri, 13 Mar 2026 04:35:13 GMT
• Title: From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space
• Abstract summary: Group Relative Policy Optimization (GRPO) has emerged as a powerful framework for preference alignment in text-to-image (T2I) flow models.<n>We propose Multi-View GRPO, a novel approach that enhances relationship exploration by augmenting the condition space.<n>MV-GRPO achieves superior alignment performance over state-of-the-art methods.
• Score: 78.36537400975298
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Group Relative Policy Optimization (GRPO) has emerged as a powerful framework for preference alignment in text-to-image (T2I) flow models. However, we observe that the standard paradigm where evaluating a group of generated samples against a single condition suffers from insufficient exploration of inter-sample relationships, constraining both alignment efficacy and performance ceilings. To address this sparse single-view evaluation scheme, we propose Multi-View GRPO (MV-GRPO), a novel approach that enhances relationship exploration by augmenting the condition space to create a dense multi-view reward mapping. Specifically, for a group of samples generated from one prompt, MV-GRPO leverages a flexible Condition Enhancer to generate semantically adjacent yet diverse captions. These captions enable multi-view advantage re-estimation, capturing diverse semantic attributes and providing richer optimization signals. By deriving the probability distribution of the original samples conditioned on these new captions, we can incorporate them into the training process without costly sample regeneration. Extensive experiments demonstrate that MV-GRPO achieves superior alignment performance over state-of-the-art methods.

Related papers

• PAPT++: Risk-Aware Adversarial Tuning and Generation for Single Domain Generalization [99.33114297216183]
  Single domain generalization aims to learn a model from one labeled source domain that generalizes to unseen target domains.<n>We introduce PAPT++, a risk-aware adversarial generation-training framework for single domain generalization.
  arXiv  Detail & Related papers  (2026-09-04T07:54:30Z)
• Semi-Supervised Adaptation of Vision-Language Models for Image Classification [2.526705939946147]
  Self-Evolutionary CLIP (SE- CLIP) is a semi-supervised framework for label mining in scene classification.<n>Results on the UCM and NWPU benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches.
  arXiv  Detail & Related papers  (2026-08-26T07:54:50Z)
• N-GRPO: Embedding-Level Neighbor Mixing for Enhanced Policy Optimization [55.14402862283128]
  We introduce N-GRPO, a novel exploration strategy integrated into the Group Relative Policy Optimization framework.<n>Rather than relying on token-level sampling or native embedding-level noise, our approach leverages Semantic Neighbor Mixing.<n>N-GRPO achieves consistent improvements over strong baselines on math reasoning benchmarks and also exhibits robust generalization capabilities on out-of-distribution tasks.
  arXiv  Detail & Related papers  (2026-06-09T12:21:27Z)
• UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation [51.41441081823758]
  Unified models capable of interleaved generation have emerged as a promising paradigm.<n>We propose a unified reinforcement learning framework tailored for interleaved generation.<n>Our experiments demonstrate that this unified training recipe significantly enhances image generation quality through reasoning.
  arXiv  Detail & Related papers  (2026-03-24T17:59:17Z)
• Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards [16.135177543347773]
  Autoregressive (AR) models are highly effective for image generation, yet their standard maximum-likelihood estimation training lacks direct optimization for sample quality and diversity.<n>We propose a lightweight RL framework that casts token-based AR inference as a Markov Decision Process, optimized via Group Relative Policy Optimization.<n>Our core contribution is the introduction of a novel distribution-level Leave-One-Out FID (LOO-FID) reward.
  arXiv  Detail & Related papers  (2026-03-24T11:28:36Z)
• Fine-tuning MLLMs Without Forgetting Is Easier Than You Think [72.59321247529975]
  We design a 2x2 experimental framework to assess model performance across in-distribution and out-of-distribution image and text inputs.<n>Our results show that appropriate regularization, such as constraining the number of trainable parameters or adopting a low learning rate, effectively prevents forgetting when dealing with out-of-distribution images.<n>We attribute this forgetting as task-specific overfitting and address this issue by introducing a data-hybrid training strategy.
  arXiv  Detail & Related papers  (2026-03-15T17:16:19Z)
• Test-Time Conditioning with Representation-Aligned Visual Features [9.262325724962485]
  We introduce Representation-Aligned Guidance (REPA-G), a framework that leverages aligned representations with rich semantic properties.<n>We steer the denoising process toward a conditioned representation extracted from a pre-trained feature extractor.<n>Our method provides versatile control at multiple scales, ranging from fine-grained texture matching via single patches to broad semantic guidance.
  arXiv  Detail & Related papers  (2026-02-03T17:15:03Z)
• Know Your Step: Faster and Better Alignment for Flow Matching Models via Step-aware Advantages [6.470160796651034]
  We propose a novel framework for training flow matching text to image models into efficient few step generators well aligned with human preferences.<n>We show that TAFS GRPO achieves strong performance in few step text to image generation and significantly improves the alignment of generated images with human preferences.
  arXiv  Detail & Related papers  (2026-02-02T03:32:00Z)
• Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation [60.33386541343322]
  We propose a Multimodal Large Language Models framework that integrates Hardness-aware and Noise-regularized preference optimization for Recommendation (HaNoRec)<n>Specifically, HaNoRec dynamically adjusts optimization weights based on both the estimated hardness of each training sample and the policy model's real-time responsiveness.
  arXiv  Detail & Related papers  (2025-11-24T04:10:46Z)
• Sample By Step, Optimize By Chunk: Chunk-Level GRPO For Text-to-Image Generation [29.015994347609936]
  Group Relative Policy Optimization (GRPO) has shown strong potential for flow-matching-based text-to-image (T2I) generation.<n>We argue that shifting the optimization paradigm from the step level to the chunk level can effectively alleviate these issues.<n>Chunk-GRPO is the first chunk-level GRPO-based approach for T2I generation.
  arXiv  Detail & Related papers  (2025-10-24T15:50:36Z)
• Incomplete Multi-view Clustering via Hierarchical Semantic Alignment and Cooperative Completion [13.39263294343983]
  This paper proposes a novel incomplete multi-view clustering framework based on Hierarchical Semantic Alignment and Cooperative Completion (HSACC)<n>HSACC achieves robust cross-view fusion through a dual-level semantic space design.<n> Experimental results demonstrate that HSACC significantly outperforms state-of-the-art methods on five benchmark datasets.
  arXiv  Detail & Related papers  (2025-10-14T02:58:10Z)
• Towards Reliable and Holistic Visual In-Context Learning Prompt Selection [82.23704441763651]
  Visual In-Context Learning (VICL) has emerged as a prominent approach for adapting visual foundation models to novel tasks.<n>VICL methods, such as Partial2Global and VPR, are grounded in the similarity-priority assumption that images more visually similar to a query image serve as better in-context examples.<n>This paper introduces an enhanced variant of Partial2Global designed for reliable and holistic selection of in-context examples in VICL.
  arXiv  Detail & Related papers  (2025-09-30T09:23:12Z)
• Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization [68.64764778089229]
  We propose MISP-DPO, the first framework to incorporate multiple, semantically diverse negative images in multimodal DPO.<n>Our method embeds prompts and candidate images in CLIP space and applies a sparse autoencoder to uncover semantic deviations into interpretable factors.<n>Experiments across five benchmarks demonstrate that MISP-DPO consistently improves multimodal alignment over prior methods.
  arXiv  Detail & Related papers  (2025-09-30T03:24:09Z)
• Deep Incomplete Multi-view Clustering with Cross-view Partial Sample and Prototype Alignment [50.82982601256481]
  We propose a Cross-view Partial Sample and Prototype Alignment Network (CPSPAN) for Deep Incomplete Multi-view Clustering. Unlike existing contrastive-based methods, we adopt pair-observed data alignment as 'proxy supervised signals' to guide instance-to-instance correspondence construction.
  arXiv  Detail & Related papers  (2023-03-28T02:31:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.