ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL

• URL: http://arxiv.org/abs/2505.24875v1
• Date: Fri, 30 May 2025 17:59:48 GMT
• Title: ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL
• Authors: Yu Zhang, Yunqi Li, Yifan Yang, Rui Wang, Yuqing Yang, Dai Qi, Jianmin Bao, Dongdong Chen, Chong Luo, Lili Qiu,
• Abstract summary: Chain-of-thought reasoning and reinforcement learning have driven breakthroughs in NLP.<n>We introduce ReasonGen-R1, a framework that imbues an autoregressive image generator with explicit text-based "thinking" skills.<n>We show that ReasonGen-R1 consistently outperforms strong baselines and prior state-of-the-art models.
• Score: 54.100889131719626
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although chain-of-thought reasoning and reinforcement learning (RL) have driven breakthroughs in NLP, their integration into generative vision models remains underexplored. We introduce ReasonGen-R1, a two-stage framework that first imbues an autoregressive image generator with explicit text-based "thinking" skills via supervised fine-tuning on a newly generated reasoning dataset of written rationales, and then refines its outputs using Group Relative Policy Optimization. To enable the model to reason through text before generating images, We automatically generate and release a corpus of model crafted rationales paired with visual prompts, enabling controlled planning of object layouts, styles, and scene compositions. Our GRPO algorithm uses reward signals from a pretrained vision language model to assess overall visual quality, optimizing the policy in each update. Evaluations on GenEval, DPG, and the T2I benchmark demonstrate that ReasonGen-R1 consistently outperforms strong baselines and prior state-of-the-art models. More: aka.ms/reasongen.

Related papers

• Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation [66.73899356886652]
  We build an image tokenizer directly atop pre-trained vision foundation models.<n>Our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality.<n>It further boosts autoregressive (AR) generation -- achieving a gFID of 2.07 on ImageNet benchmarks.
  arXiv  Detail & Related papers  (2025-07-11T09:32:45Z)
• GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning [47.592351387052545]
  GoT-R1 is a framework that applies reinforcement learning to enhance semantic-spatial reasoning in visual generation.<n>We propose a dual-stage multi-dimensional reward framework that leverages MLLMs to evaluate both the reasoning process and final output.<n> Experimental results demonstrate significant improvements on T2I-CompBench benchmark.
  arXiv  Detail & Related papers  (2025-05-22T17:59:58Z)
• Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO [68.44918104224818]
  Autoregressive image generation presents unique challenges distinct from Chain-of-Thought (CoT) reasoning.<n>This study provides the first comprehensive investigation of the GRPO and DPO algorithms in autoregressive image generation.<n>Our findings reveal that GRPO and DPO exhibit distinct advantages, and crucially, that reward models possessing stronger intrinsic generalization capabilities potentially enhance the generalization potential of the applied RL algorithms.
  arXiv  Detail & Related papers  (2025-05-22T17:59:49Z)
• DanceGRPO: Unleashing GRPO on Visual Generation [36.36813831536346]
  This paper introduces DanceGRPO, the first unified framework to adapt Group Relative Policy Optimization to visual generation paradigms.<n>We show consistent and substantial improvements, which outperform baselines by up to 181% on benchmarks such as HPS-v2.1, CLIP Score, VideoAlign, and GenEval.<n>Our results establish DanceGRPO as a robust and versatile solution for scaling Reinforcement Learning from Human Feedback tasks in visual generation, offering new insights into harmonizing reinforcement learning and visual synthesis.
  arXiv  Detail & Related papers  (2025-05-12T17:59:34Z)
• Autoregressive Image Generation with Vision Full-view Prompt [18.569610688433745]
  We propose Vision Full-view prompt (VF prompt) to enhance autoregressive image generation.<n>Inspired by prompt engineering from the field of NLP, we propose Vision Full-view prompt (VF prompt) to enhance autoregressive image generation.
  arXiv  Detail & Related papers  (2025-02-24T08:44:01Z)
• Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step [77.86514804787622]
  Chain-of-Thought (CoT) reasoning has been extensively explored in large models to tackle complex understanding tasks.<n>We provide the first comprehensive investigation of the potential of CoT reasoning to enhance autoregressive image generation.<n>We propose the Potential Assessment Reward Model (PARM) and PARM++, specialized for autoregressive image generation.
  arXiv  Detail & Related papers  (2025-01-23T18:59:43Z)
• Randomized Autoregressive Visual Generation [26.195148077398223]
  This paper presents Randomized AutoRegressive modeling (RAR) for visual generation. RAR sets a new state-of-the-art performance on the image generation task while maintaining full compatibility with language modeling frameworks. On the ImageNet-256 benchmark, RAR achieves an FID score of 1.48, not only surpassing prior state-the-art autoregressive image generators but also outperforming leading diffusion-based and masked transformer-based methods.
  arXiv  Detail & Related papers  (2024-11-01T17:59:58Z)
• CAR: Controllable Autoregressive Modeling for Visual Generation [100.33455832783416]
  Controllable AutoRegressive Modeling (CAR) is a novel, plug-and-play framework that integrates conditional control into multi-scale latent variable modeling. CAR progressively refines and captures control representations, which are injected into each autoregressive step of the pre-trained model to guide the generation process. Our approach demonstrates excellent controllability across various types of conditions and delivers higher image quality compared to previous methods.
  arXiv  Detail & Related papers  (2024-10-07T00:55:42Z)
• A Generic Approach for Enhancing GANs by Regularized Latent Optimization [79.00740660219256]
  We introduce a generic framework called em generative-model inference that is capable of enhancing pre-trained GANs effectively and seamlessly. Our basic idea is to efficiently infer the optimal latent distribution for the given requirements using Wasserstein gradient flow techniques.
  arXiv  Detail & Related papers  (2021-12-07T05:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.