CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception

• URL: http://arxiv.org/abs/2605.23655v1
• Date: Fri, 22 May 2026 14:07:44 GMT
• Title: CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
• Abstract summary: We introduce CVSearch, a training-free adaptive framework that schedules search strategies via an Assess-then-Search workflow.<n>CVSearch first invokes expert-assisted search when global information is insufficient, and only triggers a novel semantic-aware scanning mechanism upon failure.<n>Experiments on HR benchmarks demonstrate that CVSearch achieves state-of-the-art accuracy while substantially improving search efficiency.
• Score: 41.63801808635565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-resolution (HR) image perception presents a key bottleneck for multimodal large language models (MLLMs). While visual search offers a promising solution, existing methods struggle with the trade-off between coverage and efficiency. Visual expert-assisted search is efficient but prone to blind spots when proposals fail, whereas scan-based search guarantees coverage at the cost of computational redundancy and semantic fragmentation. To address this dilemma, we introduce CVSearch, a training-free adaptive framework that dynamically schedules search strategies via an Assess-then-Search workflow. Specifically, CVSearch first invokes expert-assisted search when global information is insufficient, and only triggers a novel semantic-aware scanning mechanism upon failure. Distinct from rigid grid partitioning, this efficient scanning paradigm incorporates Semantic Guided Adaptive Patching to decompose images into semantically consistent regions, effectively mitigating object fragmentation. Furthermore, we devise a Dynamic Bottom-Up Search strategy driven by a Visual Complexity prior to enable efficient and precise iterative exploration of local details. Extensive experiments on HR benchmarks demonstrate that CVSearch achieves state-of-the-art accuracy while substantially improving search efficiency. Code is released at https://github.com/liliupeng28/ICML26-CVSearch.

Related papers

• Fast-then-Fine: A Two-Stage Framework with Multi-Granular Representation for Cross-Modal Retrieval in Remote Sensing [13.181195595944603]
  Remote sensing (RS) image-text retrieval plays a critical role in understanding massive RS imagery.<n>Existing methods either rely on complex cross-modal interactions that lead to low retrieval efficiency, or depend on large-scale vision-language model pre-training.<n>We propose a fast-then-fine (FTF) two-stage retrieval framework that decomposes retrieval into a text-agnostic recall stage and a text-guided rerank stage for fine-grained alignment.
  arXiv  Detail & Related papers  (2026-04-22T10:50:38Z)
• Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models [79.77807330964576]
  Vision-DeepResearch systems use search engines for complex visual-textual fact-finding.<n>Existing benchmarks are not visual search-centric.<n>We construct the Vision-DeepResearch benchmark (VDR-Bench) comprising 2,000 VQA instances.
  arXiv  Detail & Related papers  (2026-02-02T14:53:11Z)
• Pixel-Grounded Retrieval for Knowledgeable Large Multimodal Models [58.46663983451155]
  PixSearch is an end-to-end Segmenting Large Multimodal Model (LMM) that unifies region-level perception and retrieval-augmented reasoning.<n>During encoding, PixSearch emits search> tokens to trigger retrieval, selects query modalities (text, image, or region), and generates pixel-level masks that directly serve as visual queries.<n>On egocentric and entity-centric VQA benchmarks, PixSearch substantially improves factual consistency and generalization.
  arXiv  Detail & Related papers  (2026-01-27T00:46:08Z)
• Generalized Contrastive Learning for Universal Multimodal Retrieval [53.70202081784898]
  Cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality.<n>This paper proposes Generalized Contrastive Learning (GCL), a novel loss formulation that improves multimodal retrieval performance without the need for new dataset curation.
  arXiv  Detail & Related papers  (2025-09-30T01:25:04Z)
• VectorSearch: Enhancing Document Retrieval with Semantic Embeddings and Optimized Search [1.0411820336052784]
  We propose VectorSearch, which leverages advanced algorithms, embeddings, and indexing techniques for refined retrieval. By utilizing innovative multi-vector search operations and encoding searches with advanced language models, our approach significantly improves retrieval accuracy. Experiments on real-world datasets show that VectorSearch outperforms baseline metrics.
  arXiv  Detail & Related papers  (2024-09-25T21:58:08Z)
• ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval [51.588385824875886]
  Cross-modal retrieval consists in finding images related to a given query text or vice-versa. Many recent methods proposed effective solutions to the image-text matching problem, mostly using recent large vision-language (VL) Transformer networks. This paper proposes an ALign And DIstill Network (ALADIN) to fill in the gap between effectiveness and efficiency.
  arXiv  Detail & Related papers  (2022-07-29T16:01:48Z)
• Tasks Integrated Networks: Joint Detection and Retrieval for Image Search [99.49021025124405]
  In many real-world searching scenarios (e.g., video surveillance), the objects are seldom accurately detected or annotated. We first introduce an end-to-end Integrated Net (I-Net), which has three merits. We further propose an improved I-Net, called DC-I-Net, which makes two new contributions.
  arXiv  Detail & Related papers  (2020-09-03T03:57:50Z)
• Fine-grained Visual Textual Alignment for Cross-Modal Retrieval using Transformer Encoders [14.634046503477979]
  We present a novel approach called Transformer Reasoning and Alignment Network (TERAN) TERAN enforces a fine-grained match between the underlying components of images and sentences. On the MS-COCO 1K test set, we obtain an improvement of 5.7% and 3.5% respectively on the image and the sentence retrieval tasks.
  arXiv  Detail & Related papers  (2020-08-12T11:02:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.