MolmoWeb: Open Visual Web Agent and Open Data for the Open Web

• URL: http://arxiv.org/abs/2604.08516v1
• Date: Thu, 09 Apr 2026 17:54:02 GMT
• Title: MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
• Abstract summary: MolmoWebMix is a mixture of browser task demonstrations and web-GUI perception data.<n>MolmoWeb-8B is a family of fully open multimodal web agents.<n>We will release model checkpoints, training data, code, and a unified evaluation harness to enable open research on web agents.
• Score: 60.29597961827816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Web agents--autonomous systems that navigate and execute tasks on the web on behalf of users--have the potential to transform how people interact with the digital world. However, the most capable web agents today rely on proprietary models with undisclosed training data and recipes, limiting scientific understanding, reproducibility, and community-driven progress. We believe agents for the open web should be built in the open. To this end, we introduce (1) MolmoWebMix, a large and diverse mixture of browser task demonstrations and web-GUI perception data and (2) MolmoWeb, a family of fully open multimodal web agents. Specifically, MolmoWebMix combines over 100K synthetic task trajectories from multiple complementary generation pipelines with 30K+ human demonstrations, atomic web-skill trajectories, and GUI perception data, including referring expression grounding and screenshot question answering. MolmoWeb agents operate as instruction-conditioned visual-language action policies: given a task instruction and a webpage screenshot, they predict the next browser action, requiring no access to HTML, accessibility trees, or specialized APIs. Available in 4B and 8B size, on browser-use benchmarks like WebVoyager, Online-Mind2Web, and DeepShop, MolmoWeb agents achieve state-of-the-art results outperforming similar scale open-weight-only models such as Fara-7B, UI-Tars-1.5-7B, and Holo1-7B. MolmoWeb-8B also surpasses set-of-marks (SoM) agents built on much larger closed frontier models like GPT-4o. We further demonstrate consistent gains through test-time scaling via parallel rollouts with best-of-N selection, achieving 94.7% and 60.5% pass@4 (compared to 78.2% and 35.3% pass@1) on WebVoyager and Online-Mind2Web respectively. We will release model checkpoints, training data, code, and a unified evaluation harness to enable reproducibility and accelerate open research on web agents.

Related papers

• BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes [54.96895469970198]
  BrowserForge generates web interaction data at scale by driving many browser sandboxes in parallel over the open web.<n> BrowserForge couples three components: an open-web sourcing stage that exposes the agent to hundreds of thousands of real, openly reachable websites.<n>A rule-plus-model cleaning pipeline removes failed runs and rewrites the surviving reasoning into a single unified chain-of-thought style.
  arXiv  Detail & Related papers  (2026-08-25T17:35:42Z)
• WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader [23.87868496043721]
  We propose WebGrader, a self-evolving programmatic grader that autonomously derives the required interaction flows from each website request.<n>WebGrader materializes the generated project in a live browser, grounds target actions against the source code and live DOM, and collects visual, DOM, response, and persistent-state evidence.<n>On WebGen-Bench, WebGrader trains an 8B policy to a 52.01% functional success rate, outperforming a matched appearance-plus-script reward by 7.88 points and surpassing o4-mini and DeepSeek-v4-flash.
  arXiv  Detail & Related papers  (2026-08-06T18:06:12Z)
• WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents [57.203515352080295]
  We introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution.<n>Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving.<n>As an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training.
  arXiv  Detail & Related papers  (2025-09-08T10:07:03Z)
• WebSight: A Vision-First Architecture for Robust Web Agents [0.0]
  WebSight is a vision-based web agent designed to interact with web environments purely through visual perception.<n>We introduce WebSight-7B, a fine-tuned vision-language model optimized for UI element interaction.<n>WebSight-7B achieves a top-1 accuracy of 58.84% on the Showdown Clicks benchmark, outperforming several larger generalist models.<n>WebSight and WebSight-7B establish a new standard for interpretable, robust, and efficient visual web navigation.
  arXiv  Detail & Related papers  (2025-08-23T11:02:59Z)
• MMInA: Benchmarking Multihop Multimodal Internet Agents [36.173995299002]
  We present MMInA, a multihop and multimodal benchmark to evaluate the embodied agents for compositional Internet tasks.<n>Our data includes 1,050 human-written tasks covering various domains such as shopping and travel.<n>We propose a novel protocol for evaluating an agent's progress in completing multihop tasks.
  arXiv  Detail & Related papers  (2024-04-15T17:59:50Z)
• WebLINX: Real-World Website Navigation with Multi-Turn Dialogue [23.71601972991744]
  We introduce WEBLINX - a benchmark of 100K interactions across 2300 expert demonstrations of conversational web navigation. Our benchmark covers a broad range of patterns on over 150 real-world websites and can be used to train and evaluate agents in diverse scenarios. We use the selected elements, along with screenshots and action history, to assess a variety of models for their ability to replicate human behavior when navigating the web.
  arXiv  Detail & Related papers  (2024-02-08T18:58:02Z)
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models [65.18602126334716]
  Existing web agents typically only handle one input modality and are evaluated only in simplified web simulators or static web snapshots. We introduce WebVoyager, an innovative Large Multimodal Model (LMM) powered web agent that can complete user instructions end-to-end by interacting with real-world websites. We show that WebVoyager achieves a 59.1% task success rate on our benchmark, significantly surpassing the performance of both GPT-4 (All Tools) and the WebVoyager (text-only) setups.
  arXiv  Detail & Related papers  (2024-01-25T03:33:18Z)
• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis [69.15016747150868]
  We introduce WebAgent, an agent that learns from self-experience to complete tasks on real websites. WebAgent plans ahead by decomposing instructions into canonical sub-instructions, summarizes long HTML documents into task-relevant snippets, and acts on websites. We empirically demonstrate that our modular recipe improves the success on real websites by over 50%, and that HTML-T5 is the best model to solve various HTML understanding tasks.
  arXiv  Detail & Related papers  (2023-07-24T14:56:30Z)
• Multimodal Web Navigation with Instruction-Finetuned Foundation Models [99.14209521903854]
  We study data-driven offline training for web agents with vision-language foundation models. We propose an instruction-following multimodal agent, WebGUM, that observes both webpage screenshots and HTML pages. We empirically demonstrate this recipe improves the agent's ability of grounded multimodal perception, HTML comprehension, and multi-step reasoning.
  arXiv  Detail & Related papers  (2023-05-19T17:44:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.