VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents

• URL: http://arxiv.org/abs/2603.16289v2
• Date: Wed, 18 Mar 2026 15:40:14 GMT
• Title: VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents
• Abstract summary: We introduce a new benchmark for visual-native search, VisBrowse-Bench.<n>It contains 169 VQA instances covering multiple domains.<n>It evaluates the models' visual reasoning capabilities during the search process.
• Score: 27.574370658412864
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The rapid advancement of Multimodal Large Language Models (MLLMs) has enabled browsing agents to acquire and reason over multimodal information in the real world. But existing benchmarks suffer from two limitations: insufficient evaluation of visual reasoning ability and the neglect of native visual information of web pages in the reasoning chains. To address these challenges, we introduce a new benchmark for visual-native search, VisBrowse-Bench. It contains 169 VQA instances covering multiple domains and evaluates the models' visual reasoning capabilities during the search process through multimodal evidence cross-validation via text-image retrieval and joint reasoning. These data were constructed by human experts using a multi-stage pipeline and underwent rigorous manual verification. We additionally propose an agent workflow that can effectively drive the browsing agent to actively collect and reason over visual information during the search process. We comprehensively evaluated both open-source and closed-source models in this workflow. Experimental results show that even the best-performing model, Claude-4.6-Opus only achieves an accuracy of 47.6%, while the proprietary Deep Research model, o3-deep-research only achieves an accuracy of 41.1%. The code and data can be accessed at: https://github.com/ZhengboZhang/VisBrowse-Bench

Related papers

• WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents [53.426679066400844]
  We introduce WeAgent-Harness, a multimodal agentic harness that supports native text-vision interaction and runtime recovery.<n>We develop WeAgent-MMSearch, an integrated system spanning data construction, agentic post-training, and multimodal rollout.<n>We also introduce VisTarget-Bench, a 150-task human-verified benchmark that pairs each question with a held-out target image.
  arXiv  Detail & Related papers  (2026-08-28T08:28:43Z)
• Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning [28.70926573516569]
  We propose Visual-Seeker, a visual-native multimodal deep search agent via active visual reasoning.<n>Our agent actively attends to fine-grained visual details, dynamically harvests visual evidence throughout the search process.<n>Experiments demonstrate the state-of-the-art performance across five challenging multimodal search benchmarks.
  arXiv  Detail & Related papers  (2026-06-13T10:07:32Z)
• SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain [11.397893639012272]
  We introduce SVFSearch, the first open benchmark for short-video frame search in the Chinese gaming domain.<n>We evaluate representative paradigms ranging from direct QA and RAG workflow to Plan-Act-Replan agents and learned search models.<n>Results reveal a large gap between model-only answering, practical agentic search, and oracle knowledge.
  arXiv  Detail & Related papers  (2026-05-18T07:03:48Z)
• InterLV-Search: Benchmarking Interleaved Multimodal Agentic Search [38.39587091230621]
  Existing benchmarks evaluate multimodal search and visual browsing, but visual evidence is either confined to the input or treated as an answer endpoint.<n>We introduce textbfInterLV-Search, a benchmark for Interleaved Language-Vision Agentic Search.<n>It contains 2,061 examples across three levels: active visual evidence seeking, controlled offline interleaved search, and open-web interleaved search.
  arXiv  Detail & Related papers  (2026-05-08T09:41:07Z)
• Towards Long-horizon Agentic Multimodal Search [109.0092257657625]
  We propose a novel Long-horizon MultiModal deep search framework, named LMM-Searcher, centered on a file-based visual representation mechanism.<n>We equip the agent with a tailored fetch-image tool, enabling a progressive, on-demand visual loading strategy for active perception.<n>Our method successfully scales to 100-turn search horizons, achieving state-of-the-art performance among open-source models.
  arXiv  Detail & Related papers  (2026-04-14T15:40:28Z)
• MTA-Agent: An Open Recipe for Multimodal Deep Search Agents [51.180338423927985]
  Multi-hop large language models (MLLMs) have demonstrated strong capabilities in visual understanding, yet they remain limited in complex, multi-step reasoning.<n>We propose a Multi-hop Tool-Augmented Agent for Evidence-based QA Synthesis (MTA-Agent)<n>MTA-Agent automatically selects tools and their parameters to retrieve and validate evidence from both visual and textual sources.
  arXiv  Detail & Related papers  (2026-04-07T19:01:45Z)
• MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents [57.32877731797049]
  MultiHaystack is the first benchmark designed to evaluate both retrieval and reasoning under large-scale, cross-modal conditions.<n>We find that models perform competitively when provided with the corresponding evidence, but their performance drops sharply when required to retrieve that evidence from the full corpus.
  arXiv  Detail & Related papers  (2026-03-05T21:43:02Z)
• DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories [52.57197752244638]
  We introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task.<n>Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues.<n>We construct DISBench, a challenging benchmark built on interconnected visual data.
  arXiv  Detail & Related papers  (2026-02-11T12:51:10Z)
• InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search [48.79494320593913]
  We introduce O3-Bench, a new benchmark designed to evaluate multimodal reasoning with interleaved attention to visual details.<n>O3-Bench features challenging problems that require agents to piece together subtle visual information from distinct image areas through multi-step reasoning.<n>We propose InSight-o3, a multi-agent framework consisting of a visual reasoning agent (vReasoner) and a visual search agent (vSearcher)
  arXiv  Detail & Related papers  (2025-12-21T14:23:07Z)
• MR$^2$-Bench: Going Beyond Matching to Reasoning in Multimodal Retrieval [86.35779264575154]
  Multimodal retrieval is becoming a crucial component of modern AI applications, yet its evaluation lags behind the demands of more realistic and challenging scenarios.<n>We introduce MR$2$-Bench, a reasoning-intensive benchmark for multimodal retrieval.
  arXiv  Detail & Related papers  (2025-09-30T15:09:14Z)
• MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents [78.3863007028688]
  MM-BrowseComp is a novel benchmark comprising 224 challenging, hand-crafted questions.<n>These questions often incorporate images in prompts, and crucial information encountered during the search and reasoning process may also be embedded within images or videos on webpages.<n>Our comprehensive evaluation of state-of-the-art models on MM-BrowseComp reveals that even top models like OpenAI o3 with tools achieve only 29.02% accuracy.
  arXiv  Detail & Related papers  (2025-08-14T13:46:47Z)
• Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering [60.062194349648195]
  Document Visual Question Answering (DocVQA) faces dual challenges in processing lengthy multimodal documents.<n>Current document retrieval-augmented generation (DocRAG) methods remain limited by their text-centric approaches.<n>We introduce MMDocRAG, a comprehensive benchmark featuring 4,055 expert-annotated QA pairs with multi-page, cross-modal evidence chains.
  arXiv  Detail & Related papers  (2025-05-22T09:52:57Z)
• DIME: An Online Tool for the Visual Comparison of Cross-Modal Retrieval Models [5.725477071353354]
  Cross-modal retrieval relies on accurate models to retrieve relevant results for queries across modalities such as image, text, and video. We present DIME, a modality-agnostic tool that handles multimodal datasets, trained models, and data preprocessors.
  arXiv  Detail & Related papers  (2020-10-19T16:35:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.