Advances in GRPO for Generation Models: A Survey

• URL: http://arxiv.org/abs/2603.06623v1
• Date: Sat, 21 Feb 2026 17:11:08 GMT
• Title: Advances in GRPO for Generation Models: A Survey
• Abstract summary: Flow-GRPO is a reinforcement learning framework for generative models.<n>It can be used to align generative models with human preferences and task-specific objectives.<n>This survey highlights Flow-GRPO as a general alignment framework for modern generative models.
• Score: 5.995432816974204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale flow matching models have achieved strong performance across generative tasks such as text-to-image, video, 3D, and speech synthesis. However, aligning their outputs with human preferences and task-specific objectives remains challenging. Flow-GRPO extends Group Relative Policy Optimization (GRPO) to generation models, enabling stable reinforcement learning alignment for generative systems. Since its introduction, Flow-GRPO has triggered rapid research growth, spanning methodological refinements and diverse application domains. This survey provides a comprehensive review of Flow-GRPO and its subsequent developments. We organize existing work along two primary dimensions. First, we analyze methodological advances beyond the original framework, including reward signal design, credit assignment, sampling efficiency, diversity preservation, reward hacking mitigation, and reward model construction. Second, we examine extensions of GRPO-based alignment across generative paradigms and modalities, including text-to-image, video generation, image editing, speech and audio, 3D modeling, embodied vision-language-action systems, unified multimodal models, autoregressive and masked diffusion models, and restoration tasks. By synthesizing theoretical insights and practical adaptations, this survey highlights Flow-GRPO as a general alignment framework for modern generative models and outlines key open challenges for scalable and robust reinforcement-based generation.

Related papers

• UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation [51.41441081823758]
  Unified models capable of interleaved generation have emerged as a promising paradigm.<n>We propose a unified reinforcement learning framework tailored for interleaved generation.<n>Our experiments demonstrate that this unified training recipe significantly enhances image generation quality through reasoning.
  arXiv  Detail & Related papers  (2026-03-24T17:59:17Z)
• UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? [50.92401586025528]
  Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear.<n>We introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks.
  arXiv  Detail & Related papers  (2026-03-03T18:36:16Z)
• Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances [8.56304683490938]
  Reinforcement learning offers a principled framework for optimizing non-differentiable, preference-driven, and temporally structured objectives.<n>Recent advances demonstrate its effectiveness in enhancing controllability, consistency, and human alignment across generative tasks.
  arXiv  Detail & Related papers  (2025-08-14T03:44:03Z)
• Continual Learning for Generative AI: From LLMs to MLLMs and Beyond [56.29231194002407]
  We present a comprehensive survey of continual learning methods for mainstream generative AI models.<n>We categorize these approaches into three paradigms: architecture-based, regularization-based, and replay-based.<n>We analyze continual learning setups for different generative models, including training objectives, benchmarks, and core backbones.
  arXiv  Detail & Related papers  (2025-06-16T02:27:25Z)
• Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO [68.44918104224818]
  Autoregressive image generation presents unique challenges distinct from Chain-of-Thought (CoT) reasoning.<n>This study provides the first comprehensive investigation of the GRPO and DPO algorithms in autoregressive image generation.<n>Our findings reveal that GRPO and DPO exhibit distinct advantages, and crucially, that reward models possessing stronger intrinsic generalization capabilities potentially enhance the generalization potential of the applied RL algorithms.
  arXiv  Detail & Related papers  (2025-05-22T17:59:49Z)
• DanceGRPO: Unleashing GRPO on Visual Generation [42.567425922760144]
  Reinforcement Learning (RL) has emerged as a promising approach for fine-tuning generative models.<n>Existing methods like DDPO and DPOK face fundamental limitations when scaling to large and diverse prompt sets.<n>This paper presents DanceGRPO, a framework that addresses these limitations through an innovative adaptation of Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2025-05-12T17:59:34Z)
• Generative Models in Decision Making: A Survey [63.68746774576147]
  generative models can be incorporated into decision-making systems by generating trajectories that guide agents toward high-reward state-action regions or intermediate sub-goals.<n>This paper presents a comprehensive review of the application of generative models in decision-making tasks.
  arXiv  Detail & Related papers  (2025-02-24T12:31:28Z)
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model [93.8067369210696]
  Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions. Diffusion models are one prominent type of generative model used for the generation of images through the systematic introduction of noises with repeating steps. In the era of large models, scaling up model size and the integration with large language models have further improved the performance of TTI models.
  arXiv  Detail & Related papers  (2023-09-02T03:27:20Z)
• A Generic Approach for Enhancing GANs by Regularized Latent Optimization [79.00740660219256]
  We introduce a generic framework called em generative-model inference that is capable of enhancing pre-trained GANs effectively and seamlessly. Our basic idea is to efficiently infer the optimal latent distribution for the given requirements using Wasserstein gradient flow techniques.
  arXiv  Detail & Related papers  (2021-12-07T05:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.