Towards LLM Agents for Earth Observation

• URL: http://arxiv.org/abs/2504.12110v1
• Date: Wed, 16 Apr 2025 14:19:25 GMT
• Title: Towards LLM Agents for Earth Observation
• Authors: Chia Hsiang Kao, Wenting Zhao, Shreelekha Revankar, Samuel Speas, Snehal Bhagat, Rajeev Datta, Cheng Perng Phoo, Utkarsh Mall, Carl Vondrick, Kavita Bala, Bharath Hariharan,
• Abstract summary: We introduce datasetnamenospace, a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors.<n>Using Google Earth Engine API as a tool, LLM agents can only achieve an accuracy of 33% because the code fails to run over 58% of the time.<n>We improve the failure rate for open models by fine-tuning synthetic data, allowing much smaller models to achieve comparable accuracy to much larger ones.
• Score: 49.92444022073444
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Earth Observation (EO) provides critical planetary data for environmental monitoring, disaster management, climate science, and other scientific domains. Here we ask: Are AI systems ready for reliable Earth Observation? We introduce \datasetnamenospace, a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors. Using Google Earth Engine API as a tool, LLM agents can only achieve an accuracy of 33% because the code fails to run over 58% of the time. We improve the failure rate for open models by fine-tuning synthetic data, allowing much smaller models (Llama-3.1-8B) to achieve comparable accuracy to much larger ones (e.g., DeepSeek-R1). Taken together, our findings identify significant challenges to be solved before AI agents can automate earth observation, and suggest paths forward. The project page is available at https://iandrover.github.io/UnivEarth.

Related papers

• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence. We propose a MLLM (OmniGeo) tailored to geospatial applications. By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• GeoGalactica: A Scientific Large Language Model in Geoscience [95.15911521220052]
  Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP) We specialize an LLM into geoscience, by further pre-training the model with a vast amount of texts in geoscience, as well as supervised fine-tuning (SFT) the resulting model with our custom collected instruction tuning dataset. We train GeoGalactica over a geoscience-related text corpus containing 65 billion tokens, preserving as the largest geoscience-specific text corpus. Then we fine-tune the model with 1 million pairs of instruction-tuning
  arXiv  Detail & Related papers  (2023-12-31T09:22:54Z)
• SSL4EO-L: Datasets and Foundation Models for Landsat Imagery [8.34029977985994]
  The Landsat program is the longest-running Earth observation program in history, with 50+ years of data acquisition by 8 satellites. Despite the increasing popularity of deep learning and remote sensing, the majority of researchers still use decision trees and random forests for Landsat image analysis. This paper introduces SSL4EO-L, the first ever dataset designed for Self-Supervised Learning for Earth Observation for the Landsat family of satellites.
  arXiv  Detail & Related papers  (2023-06-15T18:11:20Z)
• Identifying Exoplanets with Deep Learning. V. Improved Light Curve Classification for TESS Full Frame Image Observations [1.4060799411474627]
  This paper presents a dataset containing light curves from the Primary Mission and 1st Extended Mission full frame images and periodic signals detected via Box Least Squares. The dataset was curated using a thorough manual review process then used to train a neural network called Astronet-Triage-v2. Astronet-Triage-v2 is able to recover 3577 out of 4140 TOIs, while Astronet-Triage only recovers 3349 targets at an equal level of precision.
  arXiv  Detail & Related papers  (2023-01-03T21:58:13Z)
• EarthNets: Empowering AI in Earth Observation [24.160463837610074]
  Earth observation (EO) aims at monitoring the state of planet Earth using remote sensing data. This paper presents a comprehensive review of more than 500 publicly published datasets. We propose to measure, rank, and select datasets to build a new benchmark for model evaluation.
  arXiv  Detail & Related papers  (2022-10-10T18:09:35Z)
• Satellite Image Time Series Analysis for Big Earth Observation Data [50.591267188664666]
  This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome.
  arXiv  Detail & Related papers  (2022-04-24T15:23:25Z)
• Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions [49.79068659889639]
  Ingenuity, that just landed on Mars, will mark the beginning of a new era of exploration unhindered by traversability. We present an advanced robust monocular odometry algorithm that uses efficient optical flow tracking. We also present a novel approach to estimate the current risk of scale drift based on a principal component analysis of the relative translation information matrix.
  arXiv  Detail & Related papers  (2021-09-12T12:52:20Z)
• One Million Scenes for Autonomous Driving: ONCE Dataset [91.94189514073354]
  We introduce the ONCE dataset for 3D object detection in the autonomous driving scenario. The data is selected from 144 driving hours, which is 20x longer than the largest 3D autonomous driving dataset available. We reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset.
  arXiv  Detail & Related papers  (2021-06-21T12:28:08Z)
• EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task [12.795776149170978]
  We frame Earth surface forecasting as the task of predicting satellite imagery conditioned on future weather. EarthNet2021 is a large dataset suitable for training deep neural networks on the task. Resulting forecasts will greatly improve over the spatial resolution found in numerical models.
  arXiv  Detail & Related papers  (2021-04-16T09:47:30Z)
• Automated identification of transiting exoplanet candidates in NASA Transiting Exoplanets Survey Satellite (TESS) data with machine learning methods [1.9491825010518622]
  The AI/ML ThetaRay system is trained initially with Kepler exoplanetary data and validated with confirmed exoplanets. By the application of ThetaRay to 10,803 light curves of threshold crossing events (TCEs) produced by the TESS mission, we uncover 39 new exoplanetary candidates.
  arXiv  Detail & Related papers  (2021-02-20T12:28:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.