AstroLoc: Robust Space to Ground Image Localizer

• URL: http://arxiv.org/abs/2502.07003v1
• Date: Mon, 10 Feb 2025 20:06:14 GMT
• Title: AstroLoc: Robust Space to Ground Image Localizer
• Authors: Gabriele Berton, Alex Stoken, Carlo Masone,
• Abstract summary: We present the first pipeline capable of leveraging astronaut photos for training. AstroLoc learns a robust representation of Earth's surface features through two losses. We find that AstroLoc achieves a staggering 35% average improvement in recall@1 over previous SOTA.
• Score: 3.794096590842654
• License:
• Abstract: Astronauts take thousands of photos of Earth per day from the International Space Station, which, once localized on Earth's surface, are used for a multitude of tasks, ranging from climate change research to disaster management. The localization process, which has been performed manually for decades, has recently been approached through image retrieval solutions: given an astronaut photo, find its most similar match among a large database of geo-tagged satellite images, in a task called Astronaut Photography Localization (APL). Yet, existing APL approaches are trained only using satellite images, without taking advantage of the millions open-source astronaut photos. In this work we present the first APL pipeline capable of leveraging astronaut photos for training. We first produce full localization information for 300,000 manually weakly labeled astronaut photos through an automated pipeline, and then use these images to train a model, called AstroLoc. AstroLoc learns a robust representation of Earth's surface features through two losses: astronaut photos paired with their matching satellite counterparts in a pairwise loss, and a second loss on clusters of satellite imagery weighted by their relevance to astronaut photography via unsupervised mining. We find that AstroLoc achieves a staggering 35% average improvement in recall@1 over previous SOTA, pushing the limits of existing datasets with a recall@100 consistently over 99%. Finally, we note that AstroLoc, without any fine-tuning, provides excellent results for related tasks like the lost-in-space satellite problem and historical space imagery localization.

Related papers

• Advancing Earth Observation: A Survey on AI-Powered Image Processing in Satellites [0.0]
  Advancements in technology have led to a substantial growth in the quality & quantity of imagery captured by Earth Observation (EO) satellites. This has presented a challenge to the efficacy of the traditional workflow of transmitting this imagery to Earth for processing. An approach to addressing this issue is to use pre-trained artificial intelligence models to process images on-board the satellite.
  arXiv  Detail & Related papers  (2025-01-21T10:48:13Z)
• EarthMatch: Iterative Coregistration for Fine-grained Localization of Astronaut Photography [18.978718859476267]
  We present EarthMatch, an iterative homography estimation method that produces fine-grained localization of astronaut photographs. We prove our method's efficacy on this dataset and offer a new, fair method for image matcher comparison. Our method will enable fast and accurate localization of the 4.5 million and growing collection of astronaut photography of Earth.
  arXiv  Detail & Related papers  (2024-05-08T20:46:36Z)
• Deep Learning for Satellite Image Time Series Analysis: A Review [5.882962965835289]
  This review presents a summary of the state-of-the-art methods of modelling environmental, agricultural, and other Earth observation variables from SITS data using deep learning methods.
  arXiv  Detail & Related papers  (2024-04-05T07:44:17Z)
• EarthLoc: Astronaut Photography Localization by Indexing Earth from Space [22.398824732314015]
  Astronaut photography presents a unique Earth observations dataset with immense value for both scientific research and disaster response. Current manual localization efforts are time-consuming, motivating the need for automated solutions. We propose a novel approach - leveraging image retrieval - to address this challenge efficiently.
  arXiv  Detail & Related papers  (2024-03-11T14:30:51Z)
• Vehicle Perception from Satellite [54.07157185000604]
  The dataset is constructed based on 12 satellite videos and 14 synthetic videos recorded from GTA-V. It supports several tasks, including tiny object detection, counting and density estimation. 128,801 vehicles are annotated totally, and the number of vehicles in each image varies from 0 to 101.
  arXiv  Detail & Related papers  (2024-02-01T15:59:16Z)
• DiffusionSat: A Generative Foundation Model for Satellite Imagery [63.2807119794691]
  We present DiffusionSat, to date the largest generative foundation model trained on a collection of publicly available large, high-resolution remote sensing datasets. Our method produces realistic samples and can be used to solve multiple generative tasks including temporal generation, superresolution given multi-spectral inputs and in-painting.
  arXiv  Detail & Related papers  (2023-12-06T16:53:17Z)
• POLAR-Sim: Augmenting NASA's POLAR Dataset for Data-Driven Lunar Perception and Rover Simulation [1.9131868049527916]
  NASA's POLAR dataset contains approximately 2,600 pairs of high dynamic range stereo photos. The purpose of these photos is to spur development in robotics, AI-based perception, and autonomous navigation. This work allows anyone with a camera model to synthesize images associated with any of the 13 scenarios of the POLAR dataset.
  arXiv  Detail & Related papers  (2023-09-21T18:00:34Z)
• 6D Camera Relocalization in Visually Ambiguous Extreme Environments [79.68352435957266]
  We propose a novel method to reliably estimate the pose of a camera given a sequence of images acquired in extreme environments such as deep seas or extraterrestrial terrains. Our method achieves comparable performance with state-of-the-art methods on the indoor benchmark (7-Scenes dataset) using only 20% training data.
  arXiv  Detail & Related papers  (2022-07-13T16:40:02Z)
• Convolutional Neural Processes for Inpainting Satellite Images [56.032183666893246]
  Inpainting involves predicting what is missing based on the known pixels and is an old problem in image processing. We show ConvvNPs can outperform classical methods and state-of-the-art deep learning inpainting models on a scanline inpainting problem for LANDSAT 7 satellite images.
  arXiv  Detail & Related papers  (2022-05-24T23:29:04Z)
• VPAIR -- Aerial Visual Place Recognition and Localization in Large-scale Outdoor Environments [49.82314641876602]
  We present a new dataset named VPAIR. The dataset was recorded on board a light aircraft flying at an altitude of more than 300 meters above ground. The dataset covers a more than one hundred kilometers long trajectory over various types of challenging landscapes.
  arXiv  Detail & Related papers  (2022-05-23T18:50:08Z)
• Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image [91.29546868637911]
  This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-04-10T19:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.