Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing

• URL: http://arxiv.org/abs/2507.08575v1
• Date: Fri, 11 Jul 2025 13:23:25 GMT
• Title: Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing
• Abstract summary: Millions of biological sample records collected in the last few centuries archived in natural history collections are un-georeferenced.<n>We present preliminary experiments and results of a novel method that exploits multi-modal capabilities of recent Large Multi-Modal Models (LMM)<n>This method enables the model to visually contextualize spatial relations it reads in the locality description.
• Score: 0.5120567378386615
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Millions of biological sample records collected in the last few centuries archived in natural history collections are un-georeferenced. Georeferencing complex locality descriptions associated with these collection samples is a highly labour-intensive task collection agencies struggle with. None of the existing automated methods exploit maps that are an essential tool for georeferencing complex relations. We present preliminary experiments and results of a novel method that exploits multi-modal capabilities of recent Large Multi-Modal Models (LMM). This method enables the model to visually contextualize spatial relations it reads in the locality description. We use a grid-based approach to adapt these auto-regressive models for this task in a zero-shot setting. Our experiments conducted on a small manually annotated dataset show impressive results for our approach ($\sim$1 km Average distance error) compared to uni-modal georeferencing with Large Language Models and existing georeferencing tools. The paper also discusses the findings of the experiments in light of an LMM's ability to comprehend fine-grained maps. Motivated by these results, a practical framework is proposed to integrate this method into a georeferencing workflow.

Related papers

• GeoBenchLLM: A Comprehensive Benchmark for Evaluating LLMs on Geo-Related Tasks [1.5249435285717097]
  We present benchName, a benchmark for probing Large Language Models on geo-related tasks.<n>We leverage a careful selection of twelve publicly available datasets from diverse geo-related tasks and domains.<n>Our results show that reasoning and size have a strong impact on overall performance.
  arXiv  Detail & Related papers  (2026-08-07T16:58:04Z)
• MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training [15.675086189757769]
  MMEarth-Bench is a collection of five new multimodal environmental tasks with 12 modalities, globally distributed data, and both in- and out-of-distribution test splits.<n>We benchmark a diverse set of pretrained models and find that while (multimodal) pretraining tends to improve model robustness in limited data settings, geographic generalization abilities remain poor.<n>We propose a model-agnostic method for test-time training with multimodal reconstruction (TTT-MMR) that uses all the modalities available at test time as auxiliary tasks.
  arXiv  Detail & Related papers  (2026-02-06T00:48:19Z)
• GLiDRE: Generalist Lightweight model for Document-level Relation Extraction [1.8628821924525962]
  We introduce GLiDRE, a new compact model for document-level relation extraction, designed to work efficiently in both supervised and few-shot settings.<n> Experiments in both low-resource supervised training and few-shot meta-learning benchmarks show that our approach outperforms existing methods in data-constrained scenarios.<n>Our code will be publicly available.
  arXiv  Detail & Related papers  (2025-08-01T16:33:13Z)
• GeoMM: On Geodesic Perspective for Multi-modal Learning [55.41612200877861]
  This paper introduces geodesic distance as a novel distance metric in multi-modal learning for the first time.<n>Our approach incorporates a comprehensive series of strategies to adapt geodesic distance for the current multimodal learning.
  arXiv  Detail & Related papers  (2025-05-16T13:12:41Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• GeoJEPA: Towards Eliminating Augmentation- and Sampling Bias in Multimodal Geospatial Learning [0.0]
  We present GeoJEPA, a versatile multimodal fusion model for geospatial data built on the self-supervised Joint-Embedding Predictive Architecture.<n>We aim to eliminate the widely accepted augmentation- and sampling biases found in self-supervised geospatial representation learning.<n>The results are multimodal semantic representations of urban regions and map entities that we evaluate both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2025-02-25T22:03:28Z)
• Detecting Contextual Anomalies by Discovering Consistent Spatial Regions [5.160649627495959]
  We describe a method for modeling spatial context to enable video anomaly detection.<n>The main idea is to discover regions that share similar object-level activities by clustering joint object attributes.<n>We demonstrate this approach, using orders of magnitude fewer parameters than competing models, in the challenging spatial-context-dependent Street Scene dataset.
  arXiv  Detail & Related papers  (2025-01-14T22:33:07Z)
• MapExplorer: New Content Generation from Low-Dimensional Visualizations [60.02149343347818]
  Low-dimensional visualizations, or "projection maps," are widely used to interpret large-scale and complex datasets.<n>These visualizations not only aid in understanding existing knowledge spaces but also implicitly guide exploration into unknown areas.<n>We introduce MapExplorer, a novel knowledge discovery task that translates coordinates within any projection map into coherent, contextually aligned textual content.
  arXiv  Detail & Related papers  (2024-12-24T20:16:13Z)
• An LLM Agent for Automatic Geospatial Data Analysis [5.842462214442362]
  Large language models (LLMs) are being used in data science code generation tasks. Their application to geospatial data processing is challenging due to difficulties in incorporating complex data structures and spatial constraints. We introduce GeoAgent, a new interactive framework designed to help LLMs handle geospatial data processing more effectively.
  arXiv  Detail & Related papers  (2024-10-24T14:47:25Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• MGeo: Multi-Modal Geographic Pre-Training Method [49.78466122982627]
  We propose a novel query-POI matching method Multi-modal Geographic language model (MGeo) MGeo represents GC as a new modality and is able to fully extract multi-modal correlations for accurate query-POI matching. Our proposed multi-modal pre-training method can significantly improve the query-POI matching capability of generic PTMs.
  arXiv  Detail & Related papers  (2023-01-11T03:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.