Towards Long-horizon Agentic Multimodal Search

• URL: http://arxiv.org/abs/2604.12890v1
• Date: Tue, 14 Apr 2026 15:40:28 GMT
• Title: Towards Long-horizon Agentic Multimodal Search
• Abstract summary: We propose a novel Long-horizon MultiModal deep search framework, named LMM-Searcher, centered on a file-based visual representation mechanism.<n>We equip the agent with a tailored fetch-image tool, enabling a progressive, on-demand visual loading strategy for active perception.<n>Our method successfully scales to 100-turn search horizons, achieving state-of-the-art performance among open-source models.
• Score: 109.0092257657625
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal deep search agents have shown great potential in solving complex tasks by iteratively collecting textual and visual evidence. However, managing the heterogeneous information and high token costs associated with multimodal inputs over long horizons remains a critical challenge, as existing methods often suffer from context explosion or the loss of crucial visual signals. To address this, we propose a novel Long-horizon MultiModal deep search framework, named LMM-Searcher, centered on a file-based visual representation mechanism. By offloading visual assets to an external file system and mapping them to lightweight textual identifiers (UIDs), our approach mitigates context overhead while preserving multimodal information for future access. We equip the agent with a tailored fetch-image tool, enabling a progressive, on-demand visual loading strategy for active perception. Furthermore, we introduce a data synthesis pipeline designed to generate queries requiring complex cross-modal multi-hop reasoning. Using this pipeline, we distill 12K high-quality trajectories to fine-tune Qwen3-VL-Thinking-30A3B into a specialized multimodal deep search agent. Extensive experiments across four benchmarks demonstrate that our method successfully scales to 100-turn search horizons, achieving state-of-the-art performance among open-source models on challenging long-horizon benchmarks like MM-BrowseComp and MMSearch-Plus, while also exhibiting strong generalizability across different base models. Our code will be released in https://github.com/RUCAIBox/LMM-Searcher.

Related papers

• WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents [53.426679066400844]
  We introduce WeAgent-Harness, a multimodal agentic harness that supports native text-vision interaction and runtime recovery.<n>We develop WeAgent-MMSearch, an integrated system spanning data construction, agentic post-training, and multimodal rollout.<n>We also introduce VisTarget-Bench, a 150-task human-verified benchmark that pairs each question with a held-out target image.
  arXiv  Detail & Related papers  (2026-08-28T08:28:43Z)
• DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents [21.799085797512955]
  Multimodal large language models (MLLMs) have advanced visual understanding and reasoning.<n>DeepVoyager-VL is a long-horizon multimodal deep-search framework for vision-in-the-loop search.
  arXiv  Detail & Related papers  (2026-08-03T07:36:48Z)
• OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents [50.79529228477821]
  We introduce OpenSearch-VL, a fully open-source recipe for training frontier multimodal deep search agents with agentic reinforcement learning.<n>First, we curated a dedicated pipeline to construct high-quality training data through Wikipedia path sampling, fuzzy entity rewriting, and source-anchor visual grounding.<n>Besides, we design a diverse tool environment that unifies text search, image search, OCR, cropping, sharpening, super-resolution, and perspective correction.
  arXiv  Detail & Related papers  (2026-05-06T17:50:38Z)
• MTA-Agent: An Open Recipe for Multimodal Deep Search Agents [51.180338423927985]
  Multi-hop large language models (MLLMs) have demonstrated strong capabilities in visual understanding, yet they remain limited in complex, multi-step reasoning.<n>We propose a Multi-hop Tool-Augmented Agent for Evidence-based QA Synthesis (MTA-Agent)<n>MTA-Agent automatically selects tools and their parameters to retrieve and validate evidence from both visual and textual sources.
  arXiv  Detail & Related papers  (2026-04-07T19:01:45Z)
• VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning [22.27364585438247]
  VSearcher is a multimodal search agent capable of long-horizon, multi-turn tool use in real-world web environments.<n>We introduce Iterative Injection Data Synthesis pipeline to generate large-scale, complex multimodal QA questions.<n>We then adopt an SFT-then-RL training pipeline to turn base multimodal models to agent capable of multi-turn tool calling in real-world web environments.
  arXiv  Detail & Related papers  (2026-03-03T09:33:22Z)
• DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories [52.57197752244638]
  We introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task.<n>Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues.<n>We construct DISBench, a challenging benchmark built on interconnected visual data.
  arXiv  Detail & Related papers  (2026-02-11T12:51:10Z)
• WideSeek: Advancing Wide Research via Multi-Agent Scaling [29.02742625120584]
  Wide Research is a paradigm essential for synthesizing and synthesizing comprehensive information under complex constraints in parallel.<n>We take a deep dive into Wide Research from two perspectives: Data Pipeline and Agent Optimization.<n>First, we produce WideSeekBench, a benchmark constructed via a rigorous multi-phase data pipeline to ensure diversity across the target information volume.<n>Second, we introduce WideSeek, a dynamic hierarchical multi-agent architecture that can autonomously fork parallel sub-agents based on task requirements.
  arXiv  Detail & Related papers  (2026-02-02T18:32:48Z)
• MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents [44.63565009665076]
  We introduce MMSearch-Plus, a 311-task benchmark that enforces multimodal understanding.<n>We provide a model-agnostic agent framework with standard browsing tools and a set-of-mark (SoM) module.<n>SoM enables provenance-aware zoom-and-retrieve and improves robustness in multi-step reasoning.
  arXiv  Detail & Related papers  (2025-08-29T09:58:27Z)
• MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents [78.3863007028688]
  MM-BrowseComp is a novel benchmark comprising 224 challenging, hand-crafted questions.<n>These questions often incorporate images in prompts, and crucial information encountered during the search and reasoning process may also be embedded within images or videos on webpages.<n>Our comprehensive evaluation of state-of-the-art models on MM-BrowseComp reveals that even top models like OpenAI o3 with tools achieve only 29.02% accuracy.
  arXiv  Detail & Related papers  (2025-08-14T13:46:47Z)
• MMSearch-R1: Incentivizing LMMs to Search [49.889749277236376]
  We present MMSearch-R1, the first end-to-end reinforcement learning framework that enables on-demand, multi-turn search in real-world Internet environments.<n>Our framework integrates both image and text search tools, allowing the model to reason about when and how to invoke them guided by an outcome-based reward with a search penalty.
  arXiv  Detail & Related papers  (2025-06-25T17:59:42Z)
• Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answering [60.062194349648195]
  Document Visual Question Answering (DocVQA) faces dual challenges in processing lengthy multimodal documents.<n>Current document retrieval-augmented generation (DocRAG) methods remain limited by their text-centric approaches.<n>We introduce MMDocRAG, a comprehensive benchmark featuring 4,055 expert-annotated QA pairs with multi-page, cross-modal evidence chains.
  arXiv  Detail & Related papers  (2025-05-22T09:52:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.