Video-Based Reward Modeling for Computer-Use Agents

• URL: http://arxiv.org/abs/2603.10178v1
• Date: Tue, 10 Mar 2026 19:17:22 GMT
• Title: Video-Based Reward Modeling for Computer-Use Agents
• Abstract summary: We study reward modeling from execution video: a sequence of sequences from an agent trajectory independent of the agent's internal reasoning or actions.<n>We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets.<n>Building on these components, we fine-tune an Execution Video Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success.
• Score: 40.27314571412647
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions. Although video-execution modeling is method-agnostic, it presents key challenges, including highly redundant layouts and subtle, localized cues that determine success. We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets. We further propose adversarial instruction translation to synthesize negative samples with step-level annotations. To enable learning from long, high-resolution execution videos, we design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes. Building on these components, we fine-tune an Execution Video Reward Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success. Our ExeVRM 8B achieves 84.7% accuracy and 87.7% recall on video-execution assessment, outperforming strong proprietary models such as GPT-5.2 and Gemini-3 Pro across Ubuntu, macOS, Windows, and Android, while providing more precise temporal attribution. These results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs.

Related papers

• From Evaluation to Enhancement: Benchmarking and Improving Think-with-Video Reasoning for Video Generative Models [42.398664944109676]
  We introduce VWG-Bench (Video World Generalist Benchmark), a comprehensive benchmark spanning 9 reasoning dimensions and 38 fine-grained tasks.<n>We propose Vid-PRE (Video Reason Prompter and Enhancer), a model-agnostic prompt rewriter that offloads the cognitive burden of reasoning to a dedicated VLM.
  arXiv  Detail & Related papers  (2026-09-10T08:40:58Z)
• CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents [17.596111848044476]
  We introduce CUA-Suite, a large-scale ecosystem of expert video demonstrations and dense annotations for professional desktop computer-use agents.<n>At its core is VideoCUA, which provides approximately 10,000 human-demonstrated tasks across 87 diverse applications.<n> CUA-Suite further provides two complementary resources: UI-Vision, a rigorous benchmark for evaluating grounding and planning capabilities in CUAs, and GroundCUA, a large-scale grounding dataset with 56K annotated screenshots.
  arXiv  Detail & Related papers  (2026-03-25T15:52:56Z)
• UniVBench: Towards Unified Evaluation for Video Foundation Models [29.73247324829126]
  Video foundation models aim to integrate video understanding, generation, editing, and instruction following within a single framework.<n>We introduce UniVBench, a benchmark for evaluating video foundation models across four core abilities.<n>Our benchmark substantially expands the complexity of evaluation by incorporating 200 high-quality, diverse and multi-shot videos.
  arXiv  Detail & Related papers  (2026-02-25T12:08:53Z)
• VDOT: Efficient Unified Video Creation via Optimal Transport Distillation [70.02065520468726]
  We propose an efficient unified video creation model, named VDOT.<n>We employ a novel computational optimal transport (OT) technique to optimize the discrepancy between the real and fake score distributions.<n>To support training unified video creation models, we propose a fully automated pipeline for video data annotation and filtering.
  arXiv  Detail & Related papers  (2025-12-07T11:31:00Z)
• VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos [62.29924199978745]
  VideoAgentTrek is a scalable pipeline that automatically mines training data from publicly available screen-recorded videos at web scale.<n>Our approach addresses a key challenge: raw videos contain implicit demonstrations but lack explicit action labels.<n>applied to 39,000 YouTube tutorial videos, our pipeline generates 1.52 million interaction steps automatically.
  arXiv  Detail & Related papers  (2025-10-22T11:25:48Z)
• CAViAR: Critic-Augmented Video Agentic Reasoning [90.48729440775223]
  We ask: can perception capabilities be leveraged to perform more complex video reasoning?<n>We develop a large language model agent given access to video modules as subagents or tools.<n>We show that the combination of our agent and critic achieve strong performance on datasets.
  arXiv  Detail & Related papers  (2025-09-09T17:59:39Z)
• Learning from Streaming Video with Orthogonal Gradients [62.51504086522027]
  We address the challenge of representation learning from a continuous stream of video as input, in a self-supervised manner.<n>This differs from the standard approaches to video learning where videos are chopped and shuffled during training in order to create a non-redundant batch.<n>We demonstrate the drop in performance when moving from shuffled to sequential learning on three tasks.
  arXiv  Detail & Related papers  (2025-04-02T17:59:57Z)
• VideoSAVi: Self-Aligned Video Language Models without Human Supervision [0.6854849895338531]
  VideoSAVi is a self-training pipeline that enables Video-LLMs to learn from video content without external supervision.<n>Our approach includes a self-critiquing mechanism that identifies reasoning errors in the model's initial responses.<n>VideoSAVi delivers significant improvements across multiple benchmarks.
  arXiv  Detail & Related papers  (2024-12-01T00:33:05Z)
• Look, Remember and Reason: Grounded reasoning in videos with language models [5.3445140425713245]
  Multi-temporal language models (LM) have recently shown promising performance in high-level reasoning tasks on videos. We propose training an LM end-to-end on low-level surrogate tasks, including object detection, re-identification, tracking, to endow the model with the required low-level visual capabilities. We demonstrate the effectiveness of our framework on diverse visual reasoning tasks from the ACRE, CATER, Something-Else and STAR datasets.
  arXiv  Detail & Related papers  (2023-06-30T16:31:14Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.