EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration

• URL: http://arxiv.org/abs/2604.07070v2
• Date: Mon, 13 Apr 2026 06:23:26 GMT
• Title: EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration
• Abstract summary: We introduce EVGeoQA, a novel benchmark built upon Electric Vehicle (EV) charging scenarios.<n>Each query in EVGeoQA is explicitly bound to a user's real-time coordinate and integrates the dual objectives of a charging necessity and a co-located activity preference.<n>Our experiments reveal that while LLMs successfully utilize tools to address sub-tasks, they struggle with long-range spatial exploration.
• Score: 3.059053384364342
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Large Language Models (LLMs) demonstrate remarkable reasoning capabilities, their potential for purpose-driven exploration in dynamic geo-spatial environments remains under-investigated. Existing Geo-Spatial Question Answering (GSQA) benchmarks predominantly focus on static retrieval, failing to capture the complexity of real-world planning that involves dynamic user locations and compound constraints. To bridge this gap, we introduce EVGeoQA, a novel benchmark built upon Electric Vehicle (EV) charging scenarios that features a distinct location-anchored and dual-objective design. Specifically, each query in EVGeoQA is explicitly bound to a user's real-time coordinate and integrates the dual objectives of a charging necessity and a co-located activity preference. To systematically assess models in such complex settings, we further propose GeoRover, a general evaluation framework based on a tool-augmented agent architecture to evaluate the LLMs' capacity for dynamic, multi-objective exploration. Our experiments reveal that while LLMs successfully utilize tools to address sub-tasks, they struggle with long-range spatial exploration. Notably, we observe an emergent capability: LLMs can summarize historical exploration trajectories to enhance exploration efficiency. These findings establish EVGeoQA as a challenging testbed for future geo-spatial intelligence. The dataset and prompts are available at https://github.com/kg-bnu/EVGeoQA.

Related papers

• GeoBenchLLM: A Comprehensive Benchmark for Evaluating LLMs on Geo-Related Tasks [1.5249435285717097]
  We present benchName, a benchmark for probing Large Language Models on geo-related tasks.<n>We leverage a careful selection of twelve publicly available datasets from diverse geo-related tasks and domains.<n>Our results show that reasoning and size have a strong impact on overall performance.
  arXiv  Detail & Related papers  (2026-08-07T16:58:04Z)
• Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning [72.13218601075958]
  We introduce textbfWanderBench, the first open access global geolocation benchmark for actionable geolocation reasoning in embodied scenarios.<n>We propose textbfGeoAoT (Action of Thought), a underlineGeolocation framework with underlineAction of underlineThough, which couples reasoning with embodied actions.<n>Experiments on 19 large multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments.
  arXiv  Detail & Related papers  (2026-03-11T06:24:10Z)
• GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI [52.13138825802668]
  GeoFMs are transforming Earth Observation, but evaluation lacks standardized protocols.<n> GEO-Bench-2 addresses this with a comprehensive framework spanning classification, segmentation, regression, object detection, and instance segmentation.<n>Code, data, and leaderboard for GEO-Bench-2 are publicly released under a permissive license.
  arXiv  Detail & Related papers  (2025-11-19T17:45:02Z)
• GeoX-Bench: Benchmarking Cross-View Geo-Localization and Pose Estimation Capabilities of Large Multimodal Models [78.98542840563907]
  GeoX-Bench is a comprehensive underlineBenchmark designed to explore and evaluate the capabilities of LMMs.<n>It contains 10,859 panoramic-satellite image pairs spanning 128 cities in 49 countries, along with corresponding 755,976 question-answering (QA) pairs.<n>Based on GeoX-Bench, we evaluate the capabilities of 25 state-of-the-art LMMs on cross-view geo-localization and pose estimation tasks.
  arXiv  Detail & Related papers  (2025-11-17T11:19:07Z)
• GeoExplorer: Active Geo-localization with Curiosity-Driven Exploration [24.01750902074338]
  Active Geo-localization (AGL) is the task of localizing a goal within a predefined search area.<n>Current methods approach AGL as a goal-reaching reinforcement learning problem with a distance-based reward.<n>We propose GeoExplorer, an AGL agent that incorporates curiosity-driven exploration through intrinsic rewards.
  arXiv  Detail & Related papers  (2025-07-31T20:23:25Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questions [5.053463027769152]
  Spatial-RAG is a Retrieval-Augmented Generation framework designed for geospatial question answering.<n>It integrates structured spatial databases with large language models (LLMs) via a hybrid spatial retriever.<n>It formulates the answering process as a multi-objective optimization over spatial and semantic relevance.
  arXiv  Detail & Related papers  (2025-02-04T01:30:06Z)
• GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [84.86699025256705]
  We present GEOBench-VLM, a benchmark specifically designed to evaluate Vision-Language Models (VLMs) on geospatial tasks.<n>Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales.<n>We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges.
  arXiv  Detail & Related papers  (2024-11-28T18:59:56Z)
• An LLM Agent for Automatic Geospatial Data Analysis [5.842462214442362]
  Large language models (LLMs) are being used in data science code generation tasks. Their application to geospatial data processing is challenging due to difficulties in incorporating complex data structures and spatial constraints. We introduce GeoAgent, a new interactive framework designed to help LLMs handle geospatial data processing more effectively.
  arXiv  Detail & Related papers  (2024-10-24T14:47:25Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.