Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
Abstract Overview
This paper introduces Verifiable Visual Rewards (VVR), a framework for scoring image-generation outputs with deterministic program verifiers rather than learned evaluators such as detectors or vision-language models. The authors generate synthetic tasks involving colored geometric shapes, counts, and spatial relations, from which they derive both natural-language prompts and reference-free verifiers operating directly on pixels. They release VVRBench with 10,000 tasks and VVRBench-Challenge with 720 harder tasks to measure precise visual instruction following under controlled complexity. The benchmark results show that performance drops as task complexity increases, and the paper further studies reinforcement learning with VVR-based rewards (RLVVR) as a post-training method for diffusion models.
Novelty
The distinctive contribution is a programmatically verifiable reward framework for open-ended image generation, where prompt satisfaction is checked by deterministic pixel-based code rather than learned reward models. The work also pairs this reward design with procedurally generated, complexity-controlled benchmarks and training data, enabling both exact evaluation and reinforcement learning from the same verifier family.
Results
On VVRBench-Challenge, the best reported model, GPT-Image-2.5-Sunburst, achieves 21.39% accuracy, indicating that precise visual instruction following remains difficult for frontier systems. Using VVR scores as reinforcement-learning rewards raises Stable Diffusion 3.5 Medium from 2.81% to 28.27% on VVRBench with VVR-Easy training, and to 46.60% with VVR-Matched training. The paper also reports transfer beyond synthetic scenes, including improvements on several natural-prompt benchmarks and a 71.6% human preference win rate for VVR-Easy over the pretrained baseline on 160 non-VVR prompts.
Key Points
- VVR uses deterministic, reference-free pixel verifiers to evaluate whether generated images satisfy compositional constraints such as count, color, shape, and spatial relations.
- The released VVRBench benchmarks show substantial capability gaps in image generators, with failures especially concentrated in counting and object-matching constraints and with accuracy declining as structural complexity rises.
- Training with VVR-based rewards improves instruction following on benchmark tasks and transfers to broader image-generation evaluations, while also complementing existing post-training reward mixtures.