Leveraging Verifier-Based Reinforcement Learning in Image Editing

• URL: http://arxiv.org/abs/2604.27505v1
• Date: Thu, 30 Apr 2026 06:54:39 GMT
• Title: Leveraging Verifier-Based Reinforcement Learning in Image Editing
• Abstract summary: We introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) for downstream image editing.<n>Our experiments demonstrate that our Edit-RRM surpasses powerful VLMs as an editing-specific reward model.
• Score: 41.64251379221566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Reinforcement Learning from Human Feedback (RLHF) has become a pivotal paradigm for text-to-image generation, its application to image editing remains largely unexplored. A key bottleneck is the lack of a robust general reward model for all editing tasks. Existing edit reward models usually give overall scores without detailed checks, ignoring different instruction requirements and causing biased rewards. To address this, we argue that the key is to move from a simple scorer to a reasoning verifier. We introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) and then leverages it for downstream image editing. The Edit-RRM breaks instructions into distinct principles, evaluates the edited image against each principle, and aggregates these checks into an interpretable, fine-grained reward. To build such an RRM, we first apply supervised fine-tuning (SFT) as a ``cold-start'' to generate CoT reward trajectories. Then, we introduce Group Contrastive Preference Optimization (GCPO), a reinforcement learning algorithm that leverages human pairwise preference data to reinforce our pointwise RRM. After building the RRM, we use GRPO to train editing models with this non-differentiable yet powerful reward model. Extensive experiments demonstrate that our Edit-RRM surpasses powerful VLMs such as Seed-1.5-VL and Seed-1.6-VL as an editing-specific reward model, and we observe a clear scaling trend, with performance consistently improving from 3B to 7B parameters. Moreover, Edit-R1 delivers gains to editing models like FLUX.1-kontext, highlighting its effectiveness in enhancing image editing.

Related papers

• RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing [9.858046310638803]
  We propose a pairwise generative reward modeling framework that produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images.<n>We train dedicatedRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the model the rubric-based scoring, while further improves scoring through fine-level rewards.
  arXiv  Detail & Related papers  (2026-08-27T10:56:47Z)
• ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations [84.45092853572407]
  ARM is a representation-based AutoRegressive Model that unifies image understanding, generation, and editing.<n>Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment, and faithful reconstruction.<n>Finally, ARM applies reinforcement learning to optimize task-level objectives such as visual quality, instruction adherence, and edit consistency.
  arXiv  Detail & Related papers  (2026-06-09T17:59:28Z)
• MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching [91.83651402045108]
  MT-EditFlow is a flow-matching reinforcement learning framework designed to optimize reward signals for sequential image editing.<n>We show that MT-EditFlow significantly improves performance across diverse base models.
  arXiv  Detail & Related papers  (2026-06-01T09:46:10Z)
• ReasonEdit: Towards Interpretable Image Editing Evaluation via Reinforcement Learning [86.61218827780675]
  We present ReasonEdit, an evaluation tool for text-guided image editing.<n>It is trained using reward signals derived from RE-Reward and the Group Relative Policy Optimization (GRPO) algorithm.<n>It is capable of generating high-quality interpretable evaluation text, enabling more transparent and trustworthy assessment for image editing.
  arXiv  Detail & Related papers  (2026-05-08T09:23:26Z)
• Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation [67.26349227500084]
  Current reward models, which act as critics during reinforcement learning, often suffer from hallucinations and assign noisy scores.<n>We present FIRM, a comprehensive framework that develops robust reward models to provide accurate and reliable guidance for faithful image generation and editing.<n> FIRM mitigates hallucinations, establishing a new standard for fidelity and instruction adherence over existing general models.
  arXiv  Detail & Related papers  (2026-03-12T17:57:21Z)
• ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing [33.888289858260706]
  Reinforcement learning (RL) has been investigated for improving the quality of image editing.<n>RL faces three key challenges: (1) limited reasoning exploration confined to denoising, (2) biased reward fusion, and (3) unstable VLM-based instruction rewards.<n>We propose ThinkRL-Edit, a reasoning-centric RL framework that decouples visual reasoning from image synthesis.
  arXiv  Detail & Related papers  (2026-01-06T23:43:00Z)
• EditThinker: Unlocking Iterative Reasoning for Any Image Editor [72.28251670314451]
  We propose a deliberative editing framework to 'think' while they edit.<n>We train a single MLLM, EditThinker, to act as the reasoning engine of this framework.<n>We employ reinforcement learning to align the EditThinker's thinking with its editing, thereby generating more targeted instruction improvements.
  arXiv  Detail & Related papers  (2025-12-05T18:58:09Z)
• Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback [41.41713036839503]
  We introduce Edit-R1, a novel post-training framework for instruction-based image editing based on policy optimization.<n>We employ a Multimodal Large Language Model (MLLM) as a unified, training-free reward model, leveraging its output logits to provide fine-grained feedback.<n>Our framework is model-agnostic, delivering substantial performance gains when applied to diverse base models.
  arXiv  Detail & Related papers  (2025-10-19T15:38:06Z)
• EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling [71.8265422228785]
  Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been hindered by the lack of a high-fidelity, efficient reward signal.<n>We present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model.
  arXiv  Detail & Related papers  (2025-09-28T14:28:24Z)
• DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models [97.31200133440308]
  We propose using online reinforcement learning to fine-tune text-to-image models. We focus on diffusion models, defining the fine-tuning task as an RL problem. Our approach, coined DPOK, integrates policy optimization with KL regularization.
  arXiv  Detail & Related papers  (2023-05-25T17:35:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.