GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation

• URL: http://arxiv.org/abs/2605.14406v1
• Date: Thu, 14 May 2026 05:46:07 GMT
• Title: GeoViSTA: Geospatial Vision-Tabular Transformer for Multimodal Environment Representation
• Abstract summary: We propose GeoViSTA (Geospatial Vision-Tabular Transformer), a vision-tabular architecture that learns unified geospatial embeddings.<n>We train GeoViSTA with a self-supervised joint masked-autoencoding objective, forcing it to recover missing image patches.<n>Results demonstrate that jointly modeling the physical environment alongside structured socioeconomic context yields highly transferable representations.
• Score: 24.186635502118552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale pretraining on Earth observation imagery has yielded powerful representations of the natural and built environment. However, most existing geospatial foundation models do not directly model the structured socioeconomic covariates typically stored in tabular form. This modality gap limits their ability to capture the complete total environment, which is critical for reasoning about complex environmental, social, and health-related outcomes. In this work, we propose GeoViSTA (Geospatial Vision-Tabular Transformer), a vision-tabular architecture that learns unified geospatial embeddings from co-registered gridded imagery and tabular data. GeoViSTA utilizes bilateral cross-attention to exchange spatial and semantic information across modalities, guided by a geography-aware attention mechanism that aligns continuous image patches with irregular census-tract tokens. We train GeoViSTA with a self-supervised joint masked-autoencoding objective, forcing it to recover missing image patches and tabular rows using local spatial context and cross-modal cues. Empirically, GeoViSTA's unified embeddings improve linear probing performance on high-impact downstream tasks, outperforming baselines in predicting disease-specific mortality and fire hazard frequency across held-out regions. These results demonstrate that jointly modeling the physical environment alongside structured socioeconomic context yields highly transferable representations for holistic geospatial inference.

Related papers

• TerraDiT-$Ω$: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive [19.699267665705715]
  We introduce TerraDiT-$$, a unified spatial control framework that generates satellite imagery directly from any native primitive.<n>Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging.
  arXiv  Detail & Related papers  (2026-06-30T01:56:41Z)
• OSMGraphCLIP: Learning Global Location Representations from OpenStreetMap Graphs [3.351125901721929]
  OSMGraphCLIP represents geographic environments as heterogeneous graphs of typed OpenStreetMap (OSM) features.<n>A multi-scale graph encoder captures both fine-grained local structure and broader landscape composition.<n>We show that structured OSM data alone supports strong global location representations across domains.
  arXiv  Detail & Related papers  (2026-06-06T08:18:21Z)
• Spatial Representation Learning Beyond Pixels: Unifying Raster Data and Vector Semantics for Human-Centric Geospatial Foundation Models [12.200359585933766]
  Recent advances in self-supervised learning have given rise to Earth Observation Foundation Models (EOFMs)<n>This paper calls for a paradigm shift toward joint Spatial Representation Learning (SRL) in an unified embedding space that integrate perception with vector-based reasoning.<n>We contend that such integration is essential for developing next-generation geospatial AI systems capable of more accurate, interpretable, and grounded understanding of the Earth.
  arXiv  Detail & Related papers  (2026-06-01T15:21:58Z)
• Learning to Wander: Improving the Global Image Geolocation Ability of LMMs via Actionable Reasoning [72.13218601075958]
  We introduce textbfWanderBench, the first open access global geolocation benchmark for actionable geolocation reasoning in embodied scenarios.<n>We propose textbfGeoAoT (Action of Thought), a underlineGeolocation framework with underlineAction of underlineThough, which couples reasoning with embodied actions.<n>Experiments on 19 large multimodal models show that GeoAoT achieves superior fine-grained localization and stronger generalization in dynamic environments.
  arXiv  Detail & Related papers  (2026-03-11T06:24:10Z)
• Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework [13.268717213871147]
  Cross-modal Geo-localization (CMGL) matches ground-level text descriptions with geo-tagged aerial imagery.<n>We introduce CORE, the first million-scale dataset dedicated to global CMGL.<n>We propose a physical-law-aware network (PLANET) for cross-modal geo-localization.
  arXiv  Detail & Related papers  (2026-03-09T15:27:19Z)
• GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics [91.17301794848025]
  This paper presents GeoAgent, a model capable of reasoning closely with humans and deriving fine-grained address conclusions.<n>Previous RL-based methods have achieved breakthroughs in performance and interpretability but still remain concerns because of their reliance on AI-generated chain-of-thought (CoT) data and training strategies.
  arXiv  Detail & Related papers  (2026-02-13T04:48:05Z)
• GeoDiT: A Diffusion-based Vision-Language Model for Geospatial Understanding [14.436063587920005]
  We introduce GeoDiT, the first diffusion-based vision-language model tailored for the geospatial domain.<n>It achieves significant gains in image captioning, visual grounding, and multi-object detection.<n>Our work validates that aligning the generative process with the data's intrinsic structure is key to unlocking superior performance in complex geospatial analysis.
  arXiv  Detail & Related papers  (2025-12-02T07:59:46Z)
• GeoGNN: Quantifying and Mitigating Semantic Drift in Text-Attributed Graphs [59.61242815508687]
  Graph neural networks (GNNs) on text--attributed graphs (TAGs) encode node texts using pretrained language models (PLMs) and propagate these embeddings through linear neighborhood aggregation.<n>This work introduces a local PCA-based metric that measures the degree of semantic drift and provides the first quantitative framework to analyze how different aggregation mechanisms affect manifold structure.
  arXiv  Detail & Related papers  (2025-11-12T06:48:43Z)
• Enhancing Contrastive Learning for Geolocalization by Discovering Hard Negatives on Semivariograms [7.1220661738937325]
  We propose a novel spatially regularized contrastive learning strategy that integrates a semivariogram.<n>We show that explicitly modeling spatial priors improves image-based geo-localization performance.
  arXiv  Detail & Related papers  (2025-09-25T20:53:06Z)
• GAIR: Improving Multimodal Geo-Foundation Model with Geo-Aligned Implicit Representations [5.439918212297177]
  Key challenge in multimodal GeoFM development is to explicitly model geospatial relationships across modalities.<n>We propose GAIR, a novel multimodal GeoFM architecture integrating overhead RS data, street view imagery, and their geolocation metadata.<n>We evaluate GAIR across 10 geospatial tasks spanning RS image-based, SV image-based, and location embedding-based benchmarks.
  arXiv  Detail & Related papers  (2025-03-20T19:59:39Z)
• Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and Human-Like Reasoning Framework [59.42946541163632]
  We introduce a comprehensive geolocation framework with three key components.<n>GeoComp, a large-scale dataset; GeoCoT, a novel reasoning method; and GeoEval, an evaluation metric.<n>We demonstrate that GeoCoT significantly boosts geolocation accuracy by up to 25% while enhancing interpretability.
  arXiv  Detail & Related papers  (2025-02-19T14:21:25Z)
• Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework [51.26566634946208]
  We introduce smileGeo, a novel visual geo-localization framework. By inter-agent communication, smileGeo integrates the inherent knowledge of these agents with additional retrieved information. Results show that our approach significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-21T03:31:30Z)
• Space-aware Socioeconomic Indicator Inference with Heterogeneous Graphs [46.52719756897067]
  We present GeoHG, the first space-aware socioeconomic indicator inference method that utilizes a heterogeneous graph-based structure to represent geospace for non-continuous inference.
  arXiv  Detail & Related papers  (2024-05-23T03:19:02Z)
• GeoNet: Benchmarking Unsupervised Adaptation across Geographies [71.23141626803287]
  We study the problem of geographic robustness and make three main contributions. First, we introduce a large-scale dataset GeoNet for geographic adaptation. Second, we hypothesize that the major source of domain shifts arise from significant variations in scene context. Third, we conduct an extensive evaluation of several state-of-the-art unsupervised domain adaptation algorithms and architectures.
  arXiv  Detail & Related papers  (2023-03-27T17:59:34Z)
• A General Purpose Neural Architecture for Geospatial Systems [142.43454584836812]
  We present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias. We envision how such a model may facilitate cooperation between members of the community.
  arXiv  Detail & Related papers  (2022-11-04T09:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.