TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?

• URL: http://arxiv.org/abs/2606.13148v1
• Date: Thu, 11 Jun 2026 10:26:03 GMT
• Title: TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?
• Abstract summary: We introduce TerraBench, a benchmark for grounded Earth-science reasoning.<n>It is the first in this space to pair process-level tool-use metrics with tolerance-aware numeric scoring.<n>The benchmark comprises 403 extensive agentic tasks across three tracks.
• Score: 32.95801052071361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Climate and environmental decision-making increasingly requires reasoning across heterogeneous inputs, including gridded physical data, satellite imagery, geospatial context, and simulator outputs. Weather and climate foundation models can forecast well, but do not reason interactively in language, while large language models (LLMs) reason in language but cannot operate directly on high-dimensional Earth-system data. As a result, real scientific workflows in Earth-science remain underserved. We introduce TerraBench, a benchmark for grounded Earth-science reasoning, built on TerraAgent, a ReAct-style executable framework that interleaves reasoning, tool calls, and observations to couple LLM planning with scientific tools for environmental retrieval, geospatial processing, simulation, and artifact-backed computation. TerraBench unifies analysis of Earth observation imagery, gridded data, GIS reasoning and simulation in a single executable interface, whereas prior benchmarks isolate these capabilities into narrow individual tasks. It is also the first in this space to pair process-level tool-use metrics with tolerance-aware numeric scoring. The benchmark comprises 403 extensive agentic tasks across three tracks (Fundamentals, Simulator-Grounded, and Document-Grounded Verification) and eight application domains with 24,500 verified execution steps. These results indicate that reliable Earth-science agents must go beyond tool access to coordinate heterogeneous workflows, parameterize tools precisely, and preserve artifact provenance.

Related papers

• EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards [73.12811119273206]
  We introduce EarthVerse, a benchmark that evaluates scientific agents through package-scoped investigations.<n>Agents inspect heterogeneous event packages, choose compatible evidence, execute transparent calculations, reconcile source differences, and preserve provenance in the final answer.<n>Across systems, the best mean answer-unit accuracy is 84.65%, while the highest Strict@95 is only 34.81%.
  arXiv  Detail & Related papers  (2026-08-24T17:29:16Z)
• TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation [2.0007592415850683]
  We introduce TerraLogic, a benchmark for geospatial reasoning.<n>It comprises 545 scenario-driven, hierarchy-aware tasks.<n>We also propose HieraPlan, a tool-augmented agent that organizes toolkits into functional hierarchies and performs fault-tolerant reasoning.
  arXiv  Detail & Related papers  (2026-07-14T08:30:45Z)
• Probing Geospatial SSL Representations with Environmental Signals [51.908606610432194]
  Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task.<n>We ask a different question: do SSL representations preserve statistical associations with environmental variables that co-vary with the imaging process?<n>To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables.
  arXiv  Detail & Related papers  (2026-07-06T15:24:25Z)
• Toward a Scientific Discovery Engine for Weather and Climate Data: A Visual Analytics Workbench for Embedding-Based Exploration [0.0]
  Earth system science is producing increasingly large, high-dimensional datasets from physics based Earth system models to AI-based weather and climate models.<n> Embedding-based representations can make these data searchable through similarity search and analog retrieval, but nearest neighbors in latent space are not automatically scientifically meaningful.<n>We present an open-source visual analytics workbench for each of these steps.
  arXiv  Detail & Related papers  (2026-05-01T17:03:33Z)
• OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents [68.85365034738534]
  We introduce a unified framework for developing tool-augmented geospatial agents trained on satellite imagery, natural-language queries, and detailed reasoning traces.<n>The training pipeline relies on supervised fine-tuning over structured reasoning trajectories, aligning the model with verified multistep tool interactions.<n>The accompanying corpus comprises 14,538 training and 1,169 evaluation instances, with more than 100K reasoning steps in the training split and over 7K reasoning steps in the evaluation split.
  arXiv  Detail & Related papers  (2026-02-19T18:59:54Z)
• AgriWorld:A World Tools Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agents [17.904008870689964]
  We present a Python execution environment, AgriWorld, exposing unified tools for queries over field parcels, remote-sensing time-series analytics, crop growth simulation, and task-specific predictors (e.g. yield, stress, and disease risk)<n>On top of this environment, we design a multi-turn AgroReflective agent, that iteratively writes code, observes execution results, and refines its analysis via an execute-observe-refine loop.
  arXiv  Detail & Related papers  (2026-02-17T03:12:57Z)
• Discovering High Level Patterns from Simulation Traces [1.8964402635820152]
  We propose a natural language guided method to discover coarse-grained patterns from detailed simulation logs.<n>Specifically, we synthesize programs that operate on simulation logs and map them to a series of high level activated patterns.<n>We show, through two physics benchmarks, that this annotated representation of the simulation log is more amenable to natural language reasoning about physical systems.
  arXiv  Detail & Related papers  (2026-02-10T17:31:39Z)
• Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents [49.3216026940601]
  Earth observation is essential for understanding the states of the Earth system.<n>Recent MLLMs have advanced EO research, but they still lack the capability to tackle complex tasks that require multi-step reasoning.<n>We introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem.
  arXiv  Detail & Related papers  (2025-09-27T06:04:28Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? [58.330879414174476]
  We introduce DSBench, a benchmark designed to evaluate data science agents with realistic tasks.<n>This benchmark includes 466 data analysis tasks and 74 data modeling tasks, sourced from Eloquence and Kaggle competitions.<n>Our evaluation of state-of-the-art LLMs, LVLMs, and agents shows that they struggle with most tasks, with the best agent solving only 34.12% of data analysis tasks and achieving a 34.74% Relative Performance Gap (RPG)
  arXiv  Detail & Related papers  (2024-09-12T02:08:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.