IsoVRBench: Video Reward Modeling Without Human Feedback
Abstract Overview
The paper introduces IsoVRBench, a benchmark for video reward modeling built from controlled interventions that alter a single property of a generated video while holding the rest fixed, enabling preference labels to be derived automatically without per-pair human annotation. The benchmark comprises 8,000 pairs across eight interventions spanning motion, text alignment, visual quality, and overall quality, with human validation confirming the intended differences are perceptible. Evaluating nine video reward models and two general vision-language judges reveals that current systems separate broad generator-quality differences well but struggle with subtle temporal properties that preserve frame content. In particular, models show poor temporal-order sensitivity, and published benchmark rankings do not predict this failure. The intervention framework is also utilized to train reward models and post-train downstream video generators.
Novelty
The primary novelty is the use of controlled video interventions to generate isolated preference pairs with predetermined labels, eliminating the need for large-scale human preference collection to evaluate and train specific reward-model capabilities. This provides an easily refreshable, property-targeted benchmark and synthetic training framework that isolates individual dimensions such as frame order and motion smoothness.
Results
On IsoVRBench, the top-performing reward model achieved an average accuracy of 73.3% but only 60.8% on reordered-frame pairs compared to 100% human accuracy. Training a reward model solely on intervention-derived pairs (IsoVRM-Q2) outperformed a human-data-trained counterpart on VBench-2.0 by 3.8 points (60.1% vs. 56.3%), and combining intervention data with human labels (IsoVRM-VR) further improved VBench-2.0 to 63.5%. Furthermore, generators post-trained with intervention-supervised reward models achieved higher human preference win rates (55.4% for IsoVRM-Q2 and 51.2% for IsoVRM-VR) than the generator trained with VideoReward (43.3%).
Key Points
- IsoVRBench isolates individual video properties via controlled interventions, generating 8,000 labeled preference pairs without requiring per-pair human annotation.
- State-of-the-art video reward models and prompted vision-language judges fail noticeably on frame-order and temporal-smoothness changes despite humans finding them obvious, a weakness unpredicted by existing benchmarks.
- Intervention-derived preference supervision can effectively replace or augment human preference data, improving reward-model benchmark transfer and downstream video-generator post-training.