Scalable Reverse Image Search Engine for NASAWorldview

• URL: http://arxiv.org/abs/2108.04479v1
• Date: Tue, 10 Aug 2021 07:03:00 GMT
• Title: Scalable Reverse Image Search Engine for NASAWorldview
• Abstract summary: Researchers often spend weeks sifting through decades of unlabeled satellite imagery. We developed an interactive, scalable and fast image similarity search engine that automatically sifts through the unlabeled dataset.
• Score: 0.6524460254566903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Researchers often spend weeks sifting through decades of unlabeled satellite imagery(on NASA Worldview) in order to develop datasets on which they can start conducting research. We developed an interactive, scalable and fast image similarity search engine (which can take one or more images as the query image) that automatically sifts through the unlabeled dataset reducing dataset generation time from weeks to minutes. In this work, we describe key components of the end to end pipeline. Our similarity search system was created to be able to identify similar images from a potentially petabyte scale database that are similar to an input image, and for this we had to break down each query image into its features, which were generated by a classification layer stripped CNN trained in a supervised manner. To store and search these features efficiently, we had to make several scalability improvements. To improve the speed, reduce the storage, and shrink memory requirements for embedding search, we add a fully connected layer to our CNN make all images into a 128 length vector before entering the classification layers. This helped us compress the size of our image features from 2048 (for ResNet, which was initially tried as our featurizer) to 128 for our new custom model. Additionally, we utilize existing approximate nearest neighbor search libraries to significantly speed up embedding search. Our system currently searches over our entire database of images at 5 seconds per query on a single virtual machine in the cloud. In the future, we would like to incorporate a SimCLR based featurizing model which could be trained without any labelling by a human (since the classification aspect of the model is irrelevant to this use case).

Related papers

• Multi-Spectral Remote Sensing Image Retrieval Using Geospatial Foundation Models [0.562479170374811]
  This work proposes to use Geospatial Foundation Models, like Prithvi, for remote sensing image retrieval. We introduce two datasets to the retrieval task and observe a strong performance. Prithvi processes six bands and achieves a mean Average Precision of 97.62% on BigEarthNet-43 and 44.51% on ForestNet-12.
  arXiv  Detail & Related papers  (2024-03-04T14:00:45Z)
• CFIR: Fast and Effective Long-Text To Image Retrieval for Large Corpora [3.166549403591528]
  This paper presents a two-stage Coarse-to-Fine Index-shared Retrieval (CFIR) framework, designed for fast and effective long-text to image retrieval. CFIR surpasses existing MLLMs by up to 11.06% in Recall@1000, while reducing training and retrieval times by 68.75% and 99.79%, respectively.
  arXiv  Detail & Related papers  (2024-02-23T11:47:16Z)
• A Dynamic YOLO-Based Sequence-Matching Model for Efficient Coverless Image Steganography [12.950354250175982]
  We present an efficient coverless steganography scheme based on dynamically matched images. YOLO is employed for selecting optimal objects, and a mapping dictionary is established between these objects and scrambling factors. Under typical geometric attacks, it can recover 79.85% of secret information on average.
  arXiv  Detail & Related papers  (2024-01-22T13:35:27Z)
• SynCDR : Training Cross Domain Retrieval Models with Synthetic Data [69.26882668598587]
  In cross-domain retrieval, a model is required to identify images from the same semantic category across two visual domains. We show how to generate synthetic data to fill in these missing category examples across domains. Our best SynCDR model can outperform prior art by up to 15%.
  arXiv  Detail & Related papers  (2023-12-31T08:06:53Z)
• SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation [91.91385816767057]
  Open-vocabulary semantic segmentation strives to distinguish pixels into different semantic groups from an open set of categories. We propose a simple encoder-decoder, named SED, for open-vocabulary semantic segmentation. Our SED method achieves mIoU score of 31.6% on ADE20K with 150 categories at 82 millisecond ($ms$) per image on a single A6000.
  arXiv  Detail & Related papers  (2023-11-27T05:00:38Z)
• ZeroSearch: Local Image Search from Text with Zero Shot Learning [0.0]
  This paper presents a solution that utilizes zero shot learning to create image queries with only user provided text descriptions. The paper's primary contribution is the development of an algorithm that utilizes pre-trained models to extract features from images. The algorithm's output is a list of images sorted in descending order of similarity, helping users to locate specific images more efficiently.
  arXiv  Detail & Related papers  (2023-05-01T08:27:24Z)
• Internet Explorer: Targeted Representation Learning on the Open Web [121.02587846761627]
  Modern vision models typically rely on fine-tuning general-purpose models pre-trained on large, static datasets. We propose dynamically utilizing the Internet to quickly train a small-scale model that does extremely well on the task at hand. Our approach, called Internet Explorer, explores the web in a self-supervised manner to progressively find relevant examples that improve performance on a desired target dataset.
  arXiv  Detail & Related papers  (2023-02-27T18:59:55Z)
• Scrape, Cut, Paste and Learn: Automated Dataset Generation Applied to Parcel Logistics [58.720142291102135]
  We present a fully automated pipeline to generate a synthetic dataset for instance segmentation in four steps. We first scrape images for the objects of interest from popular image search engines. We compare three different methods for image selection: Object-agnostic pre-processing, manual image selection and CNN-based image selection.
  arXiv  Detail & Related papers  (2022-10-18T12:49:04Z)
• Compatibility-aware Heterogeneous Visual Search [93.90831195353333]
  Existing systems use the same embedding model to compute representations (embeddings) for the query and gallery images. We address two forms of compatibility: One enforced by modifying the parameters of each model that computes the embeddings, the other by modifying the architectures that compute the embeddings. Compared to ordinary (homogeneous) visual search using the largest embedding model (paragon), CMP-NAS achieves 80-fold and 23-fold cost reduction.
  arXiv  Detail & Related papers  (2021-05-13T02:30:50Z)
• BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search [100.28980854978768]
  We present Block-wisely Self-supervised Neural Architecture Search (BossNAS) We factorize the search space into blocks and utilize a novel self-supervised training scheme, named ensemble bootstrapping, to train each block separately. We also present HyTra search space, a fabric-like hybrid CNN-transformer search space with searchable down-sampling positions.
  arXiv  Detail & Related papers  (2021-03-23T10:05:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.