RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

• URL: http://arxiv.org/abs/2608.26956v2
• Date: Sun, 30 Aug 2026 03:51:39 GMT
• Title: RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing
• Abstract summary: We propose a pairwise generative reward modeling framework that produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images.<n>We train dedicatedRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the model the rubric-based scoring, while further improves scoring through fine-level rewards.
• Score: 9.858046310638803
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.

Related papers

• DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation [94.61152963112704]
  We propose DyCoRM, a dynamic, criterion-aware reward model that grounds task-relevant criteria and performs criterion-aware preference comparison.<n>We also introduce DyCoPick, which applies criterion-aware reward modeling to selecting T2I images.
  arXiv  Detail & Related papers  (2026-05-25T14:04:26Z)
• Leveraging Verifier-Based Reinforcement Learning in Image Editing [41.64251379221566]
  We introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) for downstream image editing.<n>Our experiments demonstrate that our Edit-RRM surpasses powerful VLMs as an editing-specific reward model.
  arXiv  Detail & Related papers  (2026-04-30T06:54:39Z)
• From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks [2.2483228605450956]
  Large language models have achieved remarkable progress in text generation but still struggle with generative writing tasks.<n>We propose a fine-grained evaluation pipeline WEval for writing reward models and a fine-grained reinforcement learning training framework WRL.<n>WRL constructs positive and negative samples by selectively dropping instruction requirements, allowing for more precise reward model training.
  arXiv  Detail & Related papers  (2026-04-30T05:49:29Z)
• Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation [67.26349227500084]
  Current reward models, which act as critics during reinforcement learning, often suffer from hallucinations and assign noisy scores.<n>We present FIRM, a comprehensive framework that develops robust reward models to provide accurate and reliable guidance for faithful image generation and editing.<n> FIRM mitigates hallucinations, establishing a new standard for fidelity and instruction adherence over existing general models.
  arXiv  Detail & Related papers  (2026-03-12T17:57:21Z)
• Constantly Improving Image Models Need Constantly Improving Benchmarks [109.39018167487103]
  We present ECHO, a framework for constructing benchmarks directly from real-world evidence of model use.<n>Applying this framework to GPT-4o Image Gen, we construct a dataset of over 31,000 prompts curated from social media posts.
  arXiv  Detail & Related papers  (2025-10-16T17:59:30Z)
• OneReward: Unified Mask-Guided Image Generation via Multi-Task Human Preference Learning [26.133555631867385]
  OneReward is a unified reinforcement learning framework that enhances the model's generative capabilities across multiple tasks.<n>We develop Seedream 3.0 Fill, a mask-guided generation model trained via multi-task reinforcement learning.
  arXiv  Detail & Related papers  (2025-08-28T17:59:46Z)
• ImageRAG: Dynamic Image Retrieval for Reference-Guided Image Generation [25.39019070750831]
  Diffusion models struggle to generate rare or unseen concepts.<n>We propose ImageRAG, a method that dynamically retrieves relevant images based on a given text prompt.<n>Our approach is highly adaptable and can be applied across different model types.
  arXiv  Detail & Related papers  (2025-02-13T15:36:12Z)
• EditAR: Unified Conditional Generation with Autoregressive Models [58.093860528672735]
  We propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks.<n>The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm.<n>We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods.
  arXiv  Detail & Related papers  (2025-01-08T18:59:35Z)
• Composing Ensembles of Pre-trained Models via Iterative Consensus [95.10641301155232]
  We propose a unified framework for composing ensembles of different pre-trained models. We use pre-trained models as "generators" or "scorers" and compose them via closed-loop iterative consensus optimization. We demonstrate that consensus achieved by an ensemble of scorers outperforms the feedback of a single scorer.
  arXiv  Detail & Related papers  (2022-10-20T18:46:31Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.