Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models

• URL: http://arxiv.org/abs/2605.07872v1
• Date: Fri, 08 May 2026 15:29:11 GMT
• Title: Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models
• Abstract summary: We propose a unified framework spanning benchmark design, data construction, and reward model training.<n>We introduce Video Understanding Reward Bench (VURB), a benchmark featuring 2,100 preference pairs with long chain-of-thought reasoning traces (averaging 1,143 tokens) and majority voting evaluation across general, long, and reasoning-oriented video tasks.<n>We train VideoDRM and VideoGRM, a discriminative and a generative reward model, both achieving state-of-the-art performance on VURB and VideoRewardBench.
• Score: 51.616920646492524
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal reward models have advanced substantially in text and image domains, yet progress in video understanding reward modeling remains severely limited by the lack of robust evaluation benchmarks and high-quality preference data. To address this, we propose a unified framework spanning benchmark design, data construction, and reward model training. We introduce Video Understanding Reward Bench (VURB), a benchmark featuring 2,100 preference pairs with long chain-of-thought reasoning traces (averaging 1,143 tokens) and majority voting evaluation across general, long, and reasoning-oriented video tasks. We further construct Video Understanding Preference Dataset (VUP-35K) via a fully automated pipeline, providing large-scale high-quality supervision for video reward training. Building on the data, we train VideoDRM and VideoGRM, a discriminative and a generative reward model, both achieving state-of-the-art performance on VURB and VideoRewardBench. Further analysis confirms that VUP-35K enhances both reward performance and model reasoning capability, while VideoDRM and VideoGRM yield significant gains under best-of-$N$ test-time scaling.

Related papers

• You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass [40.11359880802771]
  We present a discriminative multimodal reward model that scores all candidate responses in a single forward pass.<n>The multi-response design also yields up to $Ntimes$ wall-clock speedup and FLOPs reduction over conventional single-response scoring.<n>Our model outperforms existing larger generative and discriminative reward models.
  arXiv  Detail & Related papers  (2026-04-13T04:02:03Z)
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
  Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:59:56Z)
• ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning [103.7657839292775]
  ARM-Thinker is an Agentic multimodal Reward Model that autonomously invokes external tools to ground judgments in verifiable evidence.<n>We train ARM-Thinker with multi-stage reinforcement learning, jointly optimizing tool-calling decisions and judgment accuracy.<n>Our results demonstrate that agentic capabilities significantly enhance both accuracy and interpretability of reward models.
  arXiv  Detail & Related papers  (2025-12-04T18:59:52Z)
• Q-Save: Towards Scoring and Attribution for Generated Video Evaluation [65.83319736145869]
  We present Q-Save, a new benchmark dataset and model for holistic evaluation of AI-generated video (AIGV) quality.<n>The dataset contains near 10000 videos, each annotated with a scalar mean opinion score (MOS) and fine-grained attribution labels.<n>We propose a unified evaluation model that jointly performs quality scoring and attribution-based explanation.
  arXiv  Detail & Related papers  (2025-11-24T07:00:21Z)
• VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoning [49.610569478718226]
  multimodal reward models (RMs) have substantially improved post-training for visual generative models.<n>VideoReward Thinker (VR-Thinker) is a thinking-with-image framework that equips the RM with visual reasoning operations and a visual memory window.<n>Our approach delivers state-of-the-art accuracy among open-source models on video preference benchmarks.
  arXiv  Detail & Related papers  (2025-10-12T09:29:50Z)
• VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding [19.54215281137561]
  Multimodal reward models (MRMs) play a crucial role in the training, inference, and evaluation of Large Vision Language Models (LVLMs)<n>Existing benchmarks for evaluating MRMs in the video domain suffer from a limited number and diversity of questions.<n>We introduce VideoRewardBench, the first comprehensive benchmark covering four core aspects of video understanding.
  arXiv  Detail & Related papers  (2025-08-30T12:50:55Z)
• RewardBench: Evaluating Reward Models for Language Modeling [100.28366840977966]
  We present RewardBench, a benchmark dataset and code-base for evaluation of reward models. The dataset is a collection of prompt-chosen-rejected trios spanning chat, reasoning, and safety. On the RewardBench leaderboard, we evaluate reward models trained with a variety of methods.
  arXiv  Detail & Related papers  (2024-03-20T17:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.