STEVE: A Step Verification Pipeline for Computer-use Agent Training

• URL: http://arxiv.org/abs/2503.12532v2
• Date: Mon, 24 Mar 2025 16:33:28 GMT
• Title: STEVE: A Step Verification Pipeline for Computer-use Agent Training
• Authors: Fanbin Lu, Zhisheng Zhong, Ziqin Wei, Shu Liu, Chi-Wing Fu, Jiaya Jia,
• Abstract summary: STEVE is a step verification pipeline for computer-use agent training.<n> GPT-4o is used to verify the correctness of each step in the trajectories based on the screens before and after the action execution.<n>Our agent outperforms supervised finetuning by leveraging both positive and negative actions within a trajectory.
• Score: 84.24814828303163
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing AI agents to autonomously manipulate graphical user interfaces is a long challenging task. Recent advances in data scaling law inspire us to train computer-use agents with a scaled instruction set, yet using behavior cloning to train agents still requires immense high-quality trajectories. To meet the scalability need, we designed STEVE, a step verification pipeline for computer-use agent training. First, we establish a large instruction set for computer-use agents and collect trajectory data with some suboptimal agents. GPT-4o is used to verify the correctness of each step in the trajectories based on the screens before and after the action execution, assigning each step with a binary label. Last, we adopt the Kahneman and Tversky Optimization to optimize the agent from the binary stepwise labels. Extensive experiments manifest that our agent outperforms supervised finetuning by leveraging both positive and negative actions within a trajectory. Also, STEVE enables us to train a 7B vision-language model as a computer-use agent, achieving leading performance in the challenging live desktop environment WinAgentArena with great efficiency at a reduced cost. Code and data: https://github.com/FanbinLu/STEVE.

Related papers

• Iterative Trajectory Exploration for Multimodal Agents [69.32855772335624]
  We propose an online self-exploration method for multimodal agents, namely SPORT. SPORT operates through four iterative components: task synthesis, step sampling, step verification, and preference tuning. Evaluation in the GTA and GAIA benchmarks show that the SPORT Agent achieves 6.41% and 3.64% improvements.
  arXiv  Detail & Related papers  (2025-04-30T12:01:27Z)
• Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents [30.253353551910404]
  Computer use agents automate digital tasks by directly interacting with graphical user interfaces (GUIs) on computers and mobile devices. We introduce Agent S2, a novel compositional framework that delegates cognitive responsibilities across various generalist and specialist models. Agent S2 establishes new state-of-the-art (SOTA) performance on three prominent computer use benchmarks.
  arXiv  Detail & Related papers  (2025-04-01T15:40:27Z)
• Boosting Virtual Agent Learning and Reasoning: A Step-wise, Multi-dimensional, and Generalist Reward Model with Benchmark [72.46357004059661]
  We propose Similar, a step-wise Multi-dimensional Generalist Reward Model. It offers fine-grained signals for agent training and can choose better action for inference-time scaling. We introduce the first benchmark in the virtual agent domain for step-wise, multi-dimensional reward model training and evaluation.
  arXiv  Detail & Related papers  (2025-03-24T13:30:47Z)
• OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis [55.390060529534644]
  We propose OS-Genesis, a novel data synthesis pipeline for Graphical User Interface (GUI) agents.<n>Instead of relying on pre-defined tasks, OS-Genesis enables agents first to perceive environments and perform step-wise interactions.<n>A trajectory reward model is then employed to ensure the quality of the generated trajectories.
  arXiv  Detail & Related papers  (2024-12-27T16:21:58Z)
• AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials [53.376263056033046]
  Existing approaches rely on expensive human annotation, making them unsustainable at scale.<n>We propose AgentTrek, a scalable data synthesis pipeline that generates web agent trajectories by leveraging publicly available tutorials.<n>Our fully automated approach significantly reduces data collection costs, achieving a cost of just $0.55 per high-quality trajectory without human annotators.
  arXiv  Detail & Related papers  (2024-12-12T18:59:27Z)
• SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber World [50.937342998351426]
  Chain-of-User-Thought (COUT) is a novel embodied reasoning paradigm.<n>We introduce SmartAgent, an agent framework perceiving cyber environments and reasoning personalized requirements.<n>Our work is the first to formulate the COUT process, serving as a preliminary attempt towards embodied personalized agent learning.
  arXiv  Detail & Related papers  (2024-12-10T12:40:35Z)
• Agent S: An Open Agentic Framework that Uses Computers Like a Human [31.16046798529319]
  We present Agent S, an open agentic framework that enables autonomous interaction with computers through a Graphical User Interface (GUI) Agent S aims to address three key challenges in automating computer tasks: acquiring domain-specific knowledge, planning over long task horizons, and handling dynamic, non-uniform interfaces.
  arXiv  Detail & Related papers  (2024-10-10T17:43:51Z)
• A Zero-Shot Language Agent for Computer Control with Structured Reflection [19.526676887048662]
  Large language models (LLMs) have shown increasing capacity at planning and executing a high-level goal in a live computer environment. To perform a task, recent works often require a model to learn from trace examples of the task via either supervised learning or few/many-shot prompting. We approach this problem with a zero-shot agent that requires no given expert traces.
  arXiv  Detail & Related papers  (2023-10-12T21:53:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.