Zero-Shot Satellite Image Retrieval through Joint Embeddings: Application to Crisis Response

• URL: http://arxiv.org/abs/2605.05405v2
• Date: Fri, 08 May 2026 11:41:38 GMT
• Title: Zero-Shot Satellite Image Retrieval through Joint Embeddings: Application to Crisis Response
• Abstract summary: GeoQuery is a zero-shot retrieval system that sidesteps data and compute constraints through a two-stage semantic and visual search.<n>On 76 disaster-location queries covering UK floods, US wildfires, and US droughts, GeoQuery achieves 31.6% accuracy within 50,km.
• Score: 0.6614755043607777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic search of Earth observation archives remains challenging. Visual foundation models such as CLAY produce rich embeddings of satellite imagery but lack the natural-language grounding needed for intuitive query, and full contrastive training of a remote-sensing CLIP-style model requires paired data and compute that are unavailable at global scale. To allow natural language querying at global scales, we present GeoQuery, a zero-shot retrieval system that sidesteps data and compute constraints through a two-stage semantic and visual search, leveraging a natural language embedding of a subset (proxy) of global data. Rather than training a joint encoder, we generate language descriptions for a 100k proxy subset of global Sentinel-2 tiles and optimise the description-generation prompt so that distances in the resulting text-embedding space correlate with distances in the frozen CLAY visual-embedding space. Queries are resolved in two stages, with a text-similarity search over the proxy subset followed by a visual nearest-neighbour search over worldwide CLAY embeddings On 76 disaster-location queries covering UK floods, US wildfires, and US droughts, GeoQuery achieves 31.6\% accuracy within 50\,km, with the strongest performance on floods (50\% within 50\,km) where terrain features are well captured by RGB embeddings. Deployed within a crisis response system called \ECHO{}, GeoQuery identified vulnerable areas during Brisbane's 2025 Cyclone Alfred, with downstream flood simulations reproducing historical patterns. Prompt-aligned proxies offer a practical bridge between EO foundation models and operational retrieval when full contrastive training is out of reach.

Related papers

• Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework [13.268717213871147]
  Cross-modal Geo-localization (CMGL) matches ground-level text descriptions with geo-tagged aerial imagery.<n>We introduce CORE, the first million-scale dataset dedicated to global CMGL.<n>We propose a physical-law-aware network (PLANET) for cross-modal geo-localization.
  arXiv  Detail & Related papers  (2026-03-09T15:27:19Z)
• AIFloodSense: A Global Aerial Imagery Dataset for Semantic Segmentation and Understanding of Flooded Environments [1.381010753883328]
  We introduce AIFloodSense, a comprehensive, publicly available aerial imagery dataset comprising 470 high-resolution images from 230 distinct flood events across 64 countries and six continents.<n>Unlike prior benchmarks, AIFloodSense ensures global diversity and temporal relevance (2022-2024), supporting three complementary tasks.<n>We establish baseline benchmarks for all tasks using state-of-the-art architectures, demonstrating the dataset's complexity and its value.
  arXiv  Detail & Related papers  (2025-12-19T10:34:45Z)
• VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing [59.73939718087177]
  Single-encoder vision-language model trained contrastively to embed interleaved inputs in a unified vector space.<n>VLM2GeoVec unifies scalable retrieval with region-level spatial reasoning, enabling cohesive multimodal analysis in remote sensing.
  arXiv  Detail & Related papers  (2025-12-12T11:39:35Z)
• Scaling Image Geo-Localization to Continent Level [48.7766435870634]
  This paper introduces a hybrid approach that achieves fine-grained geo-localization across a large geographic expanse the size of a continent.<n>We leverage a proxy classification task during training to learn rich feature representations that implicitly encode precise location information.<n>Our evaluation demonstrates that our approach can localize within 200m more than 68% of queries of a dataset covering a large part of Europe.
  arXiv  Detail & Related papers  (2025-10-30T17:59:35Z)
• Empowering LLM Agents with Geospatial Awareness: Toward Grounded Reasoning for Wildfire Response [9.801192259936888]
  Existing statistical approaches often lack semantic context, generalize poorly across events, and offer limited interpretability.<n>We introduce a Geospatial Awareness Layer (GAL) that grounds LLM agents in structured earth data.<n>GAL automatically retrieves and integrates infrastructure, demographic, terrain, and weather information from external geo databases.<n>This enriched context enables agents to produce evidence-based resource-allocation recommendations.
  arXiv  Detail & Related papers  (2025-10-14T01:59:02Z)
• CLOSP: A Unified Semantic Space for SAR, MSI, and Text in Remote Sensing [20.32829825711161]
  We introduce CrisisLandMark, a new large-scale corpus of over 647,000 Sentinel-1 SAR and Sentinel-2 multispectral images.<n>We then present CLOSP, a novel framework that uses text as a bridge to align unpaired optical and SAR images into a unified embedding space.<n>Our experiments show that CLOSP achieves a new state-of-the-art, improving retrieval nDGC@1000 by 54% over existing models.
  arXiv  Detail & Related papers  (2025-07-14T15:46:56Z)
• Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions [5.053463027769152]
  Spatial-RAG is a Retrieval-Augmented Generation framework designed for geospatial question answering.<n>It integrates structured spatial databases with large language models (LLMs) via a hybrid spatial retriever.<n>It formulates the answering process as a multi-objective optimization over spatial and semantic relevance.
  arXiv  Detail & Related papers  (2025-02-04T01:30:06Z)
• Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework [51.26566634946208]
  We introduce smileGeo, a novel visual geo-localization framework. By inter-agent communication, smileGeo integrates the inherent knowledge of these agents with additional retrieved information. Results show that our approach significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-21T03:31:30Z)
• Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching [60.645802236700035]
  Navigating drones through natural language commands remains challenging due to the dearth of accessible multi-modal datasets. We introduce GeoText-1652, a new natural language-guided geo-localization benchmark. This dataset is systematically constructed through an interactive human-computer process.
  arXiv  Detail & Related papers  (2023-11-21T17:52:30Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• SIRI: Spatial Relation Induced Network For Spatial Description Resolution [64.38872296406211]
  We propose a novel relationship induced (SIRI) network for language-guided localization. We show that our method is around 24% better than the state-of-the-art method in terms of accuracy, measured by an 80-pixel radius. Our method also generalizes well on our proposed extended dataset collected using the same settings as Touchdown.
  arXiv  Detail & Related papers  (2020-10-27T14:04:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.