SearchAD: Large-Scale Rare Image Retrieval Dataset for Autonomous Driving

• URL: http://arxiv.org/abs/2604.08008v1
• Date: Thu, 09 Apr 2026 09:10:41 GMT
• Title: SearchAD: Large-Scale Rare Image Retrieval Dataset for Autonomous Driving
• Abstract summary: Retrieving rare and safety-critical driving scenarios from large-scale datasets is essential for building robust autonomous driving systems.<n>We introduce SearchAD, a large-scale rare image retrieval dataset for AD containing over 423k frames drawn from 11 established datasets.<n>It specifically targets the needle-in-a-haystack problem of locating extremely rare classes, with some appearing fewer than 50 times across the entire dataset.
• Score: 6.044776150961302
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieving rare and safety-critical driving scenarios from large-scale datasets is essential for building robust autonomous driving (AD) systems. As dataset sizes continue to grow, the key challenge shifts from collecting more data to efficiently identifying the most relevant samples. We introduce SearchAD, a large-scale rare image retrieval dataset for AD containing over 423k frames drawn from 11 established datasets. SearchAD provides high-quality manual annotations of more than 513k bounding boxes covering 90 rare categories. It specifically targets the needle-in-a-haystack problem of locating extremely rare classes, with some appearing fewer than 50 times across the entire dataset. Unlike existing benchmarks, which focused on instance-level retrieval, SearchAD emphasizes semantic image retrieval with a well-defined data split, enabling text-to-image and image-to-image retrieval, few-shot learning, and fine-tuning of multi-modal retrieval models. Comprehensive evaluations show that text-based methods outperform image-based ones due to stronger inherent semantic grounding. While models directly aligning spatial visual features with language achieve the best zero-shot results, and our fine-tuning baseline significantly improves performance, absolute retrieval capabilities remain unsatisfactory. With a held-out test set on a public benchmark server, SearchAD establishes the first large-scale dataset for retrieval-driven data curation and long-tail perception research in AD: https://iis-esslingen.github.io/searchad/

Related papers

• DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories [52.57197752244638]
  We introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task.<n>Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues.<n>We construct DISBench, a challenging benchmark built on interconnected visual data.
  arXiv  Detail & Related papers  (2026-02-11T12:51:10Z)
• ILIAS: Instance-Level Image retrieval At Scale [10.013289586440104]
  ILIAS is a new test dataset for Instance-Level Image retrieval At Scale.<n>It is designed to evaluate the ability of current and future foundation models and retrieval techniques to recognize particular objects.
  arXiv  Detail & Related papers  (2025-02-17T12:42:38Z)
• Synthesizing Efficient Data with Diffusion Models for Person Re-Identification Pre-Training [51.87027943520492]
  We present a novel paradigm Diffusion-ReID to efficiently augment and generate diverse images based on known identities. Benefiting from our proposed paradigm, we first create a new large-scale person Re-ID dataset Diff-Person, which consists of over 777K images from 5,183 identities.
  arXiv  Detail & Related papers  (2024-06-10T06:26:03Z)
• EDIS: Entity-Driven Image Search over Multimodal Web Content [95.40238328527931]
  We introduce textbfEntity-textbfDriven textbfImage textbfSearch (EDIS), a dataset for cross-modal image search in the news domain. EDIS consists of 1 million web images from actual search engine results and curated datasets, with each image paired with a textual description.
  arXiv  Detail & Related papers  (2023-05-23T02:59:19Z)
• Rethinking Benchmarks for Cross-modal Image-text Retrieval [44.31783230767321]
  Cross-modal semantic understanding and matching is a major challenge in image-text retrieval. In this paper, we review the two common benchmarks and observe that they are insufficient to assess the true capability of models on fine-grained cross-modal semantic matching. We propose a novel semi-automatic renovation approach to refine coarse-grained sentences into finer-grained ones with little human effort. The results show that even the state-of-the-art models have much room for improvement in fine-grained semantic understanding.
  arXiv  Detail & Related papers  (2023-04-21T09:07:57Z)
• LAION-5B: An open large-scale dataset for training next generation image-text models [16.129935376579326]
  We present LAION-5B, a dataset consisting of 5.85 billion CLIP-filtered image-text pairs, of which 2.32B contain English language. We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset. We also provide several nearest neighbor indices, an improved web-interface for dataset exploration and subset generation.
  arXiv  Detail & Related papers  (2022-10-16T00:08:18Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Unsupervised Pre-training for Person Re-identification [90.98552221699508]
  We present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" We make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation.
  arXiv  Detail & Related papers  (2020-12-07T14:48:26Z)
• On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances and Million-AID [57.71601467271486]
  This article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. We first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million-AID, a new large-scale benchmark dataset.
  arXiv  Detail & Related papers  (2020-06-22T17:59:00Z)
• Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval [9.922132565411664]
  We introduce the Google Landmarks dataset v2 (GLDv2), a new benchmark for large-scale, fine-grained instance recognition and image retrieval. GLDv2 is the largest such dataset to date by a large margin, including over 5M images and 200k distinct instance labels. The dataset is sourced from Wikimedia Commons, the world's largest crowdsourced collection of landmark photos.
  arXiv  Detail & Related papers  (2020-04-03T22:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.