UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics

• URL: http://arxiv.org/abs/2606.15890v1
• Date: Sun, 14 Jun 2026 16:19:21 GMT
• Title: UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics
• Abstract summary: UrbanWell is a benchmark designed to evaluate the capabilities of multimodal language (MLLMs) for urban wellbeing analytics.<n>We benchmark 15 representative MLLMs in a zero-shot setting, providing a comparative evaluation across spatial and temporal dimensions.<n>UrbanWell serves as a unified benchmark for evaluating multimodal and temporal reasoning in urban wellbeing analytics.
• Score: 18.634608060111713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding urban wellbeing from multimodal data requires integrating heterogeneous spatial and temporal signals, posing significant challenges for current multimodal large language models (MLLMs). We introduce UrbanWell, a large-scale benchmark designed to systematically evaluate the spatio-temporal reasoning capabilities of MLLMs for urban wellbeing analytics through joint modeling of satellite and street view imagery. UrbanWell spans 38 cities across multiple years and includes diverse indicators covering (1) environmental conditions (CO$_2$, NO$_2$, PM${2.5}$, and Normalized Difference Vegetation Index), (2) spatial accessibility (minimum distance to supermarkets and restaurants), (3) urban form (road length, road density, and land use), (4) urban vitality (population, economic activity diversity, and land use diversity), and (5) subjective perception attributes (e.g., safety, beauty, liveliness, wealth, and quietness). All indicators are aligned at grid level to enable standardized evaluation. Beyond static prediction, UrbanWell defines temporal reasoning tasks, including future value forecasting from historical observations and temporal trend classification. We benchmark 15 state-of-the-art representative MLLMs in a zero-shot setting, providing a comprehensive comparative evaluation across spatial and temporal dimensions. Experimental results indicate that while MLLMs capture salient spatial and perceptual cues, their performance varies substantially across heterogeneous urban indicators spanning environment and subjective perception. UrbanWell serves as a unified benchmark for evaluating multimodal spatial and temporal reasoning in urban wellbeing analytics, offering a standardized testbed for systematic assessment and future research on multimodal urban intelligence. Our codes and datasets are accessible via https://github.com/axin1301/UrbanWell-Benchmark.

Related papers

• Earth Embeddings Reveal Diverse Urban Signals from Space [17.134313498337804]
  We benchmark three Earth embedding families, AlphaEarth, Prithvi, and Clay, for urban signal prediction.<n>Using a unified supervised-learning framework, we predict 14 neighborhood-level indicators spanning crime, income, health, and travel behavior.<n>Results show that Earth embeddings capture substantial urban variation, with the highest predictive skill for outcomes tied to built-environment structure.
  arXiv  Detail & Related papers  (2026-04-03T20:58:37Z)
• UrbanMoE: A Sparse Multi-Modal Mixture-of-Experts Framework for Multi-Task Urban Region Profiling [47.568568425459716]
  We develop a benchmark for multi-task urban region profiling, featuring multi-modal features and a diverse set of strong baselines.<n>We then propose UrbanMoE, the first sparse multi-modal, multi-expert framework specifically architected to solve the multi-task challenge.<n>We conduct extensive experiments on three real-world datasets within our benchmark, where UrbanMoE consistently demonstrates superior performance over all baselines.
  arXiv  Detail & Related papers  (2026-01-30T09:25:05Z)
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks [108.15756345836901]
  We provide a comprehensive review of multimodal spatial reasoning tasks with large models.<n>We review advances in embodied AI, including vision-language navigation and action models.<n>We consider emerging modalities such as audio and egocentric video, which contribute to novel spatial understanding through new sensors.
  arXiv  Detail & Related papers  (2025-10-29T17:55:43Z)
• UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective [26.682345246235766]
  UrbanFeel comprises 14.3K carefully constructed visual questions spanning three cognitively progressive dimensions.<n> Gemini-2.5 Pro achieves the best overall performance, with its accuracy approaching human expert levels.
  arXiv  Detail & Related papers  (2025-09-26T11:38:57Z)
• HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding [120.84817886550765]
  Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos.<n>Existing human-centric benchmarks predominantly emphasize video generation quality and action recognition, while overlooking essential perceptual and cognitive abilities required in human-centered scenarios.<n>We propose a rigorously curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric video understanding.
  arXiv  Detail & Related papers  (2025-07-07T11:52:24Z)
• UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding [5.312363883238377]
  We introduce $textitUrbanLLaVA$, a multi-modal large language model to process multi-modal data simultaneously.<n>We propose a multi-stage training framework that decouples spatial reasoning enhancement from domain knowledge learning.<n> Experimental results from three cities demonstrate that $textitUrbanLLaVA$ outperforms open-source and proprietary MLLMs in both single-modal tasks and complex cross-modal tasks.
  arXiv  Detail & Related papers  (2025-06-29T13:04:27Z)
• CityLens: Benchmarking Large Language-Vision Models for Urban Socioeconomic Sensing [18.67492140450614]
  CityLens is a benchmark designed to evaluate the capabilities of large language-vision models (LLVMs) in predicting socioeconomic indicators from satellite and street view imagery.<n>We construct a multi-modal dataset covering a total of 17 globally distributed cities, spanning 6 key domains: economy, education, crime, transport, health, and environment.<n>Our results reveal that while LLVMs demonstrate promising perceptual and reasoning capabilities, they still exhibit limitations in predicting urban socioeconomic indicators.
  arXiv  Detail & Related papers  (2025-05-31T12:25:33Z)
• UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios [60.492736455572015]
  We present UrBench, a benchmark designed for evaluating LMMs in complex multi-view urban scenarios.<n>UrBench contains 11.6K meticulously curated questions at both region-level and role-level.<n>Our evaluations on 21 LMMs show that current LMMs struggle in the urban environments in several aspects.
  arXiv  Detail & Related papers  (2024-08-30T13:13:35Z)
• CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks [10.22654338686634]
  Large language models (LLMs) and vision-language models (VLMs) have become essential to ensure their real-world effectiveness and reliability.<n>The challenge in constructing a systematic evaluation benchmark for urban research lies in the diversity of urban data.<n>In this paper, we design textitCityBench, an interactive simulator based evaluation platform.
  arXiv  Detail & Related papers  (2024-06-20T02:25:07Z)
• Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks [82.82866901799565]
  We build a new set of multimodal remote sensing benchmark datasets (including hyperspectral, multispectral, SAR) for the study purpose of the cross-city semantic segmentation task. Beyond the single city, we propose a high-resolution domain adaptation network, HighDAN, to promote the AI model's generalization ability from the multi-city environments. HighDAN is capable of retaining the spatially topological structure of the studied urban scene well in a parallel high-to-low resolution fusion fashion.
  arXiv  Detail & Related papers  (2023-09-26T23:55:39Z)
• Unified Data Management and Comprehensive Performance Evaluation for Urban Spatial-Temporal Prediction [Experiment, Analysis & Benchmark] [78.05103666987655]
  This work addresses challenges in accessing and utilizing diverse urban spatial-temporal datasets. We introduceatomic files, a unified storage format designed for urban spatial-temporal big data, and validate its effectiveness on 40 diverse datasets. We conduct extensive experiments using diverse models and datasets, establishing a performance leaderboard and identifying promising research directions.
  arXiv  Detail & Related papers  (2023-08-24T16:20:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.