Geospatial Mechanistic Interpretability of Large Language Models

• URL: http://arxiv.org/abs/2505.03368v2
• Date: Mon, 12 May 2025 15:44:44 GMT
• Title: Geospatial Mechanistic Interpretability of Large Language Models
• Authors: Stef De Sabbata, Stefano Mizzaro, Kevin Roitero,
• Abstract summary: Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks.<n>Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information.
• Score: 6.0272491755196045
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks. Their ability to process and generate viable text and code has made them ubiquitous in many fields, while their deployment as knowledge bases and "reasoning" tools remains an area of ongoing research. In geography, a growing body of literature has been focusing on evaluating LLMs' geographical knowledge and their ability to perform spatial reasoning. However, very little is still known about the internal functioning of these models, especially about how they process geographical information. In this chapter, we establish a novel framework for the study of geospatial mechanistic interpretability - using spatial analysis to reverse engineer how LLMs handle geographical information. Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information - what one might call "how LLMs think about geographic information" if such phrasing was not an undue anthropomorphism. We first outline the use of probing in revealing internal structures within LLMs. We then introduce the field of mechanistic interpretability, discussing the superposition hypothesis and the role of sparse autoencoders in disentangling polysemantic internal representations of LLMs into more interpretable, monosemantic features. In our experiments, we use spatial autocorrelation to show how features obtained for placenames display spatial patterns related to their geographic location and can thus be interpreted geospatially, providing insights into how these models process geographical information. We conclude by discussing how our framework can help shape the study and use of foundation models in geography.

Related papers

• Can LLMs Learn to Map the World from Local Descriptions? [50.490593949836146]
  This study investigates whether Large Language Models (LLMs) can construct coherent global spatial cognition.<n> Experiments conducted in a simulated urban environment demonstrate that LLMs exhibit latent representations aligned with real-world spatial distributions.
  arXiv  Detail & Related papers  (2025-05-27T08:22:58Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• PEACE: Empowering Geologic Map Holistic Understanding with MLLMs [64.58959634712215]
  Geologic map, as a fundamental diagram in geology science, provides critical insights into the structure and composition of Earth's subsurface and surface.<n>Despite their significance, current Multimodal Large Language Models (MLLMs) often fall short in geologic map understanding.<n>To quantify this gap, we construct GeoMap-Bench, the first-ever benchmark for evaluating MLLMs in geologic map understanding.
  arXiv  Detail & Related papers  (2025-01-10T18:59:42Z)
• SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models [70.01883340129204]
  spatial reasoning is a crucial component of both biological and artificial intelligence. We present a comprehensive study of the capability of current state-of-the-art large language models (LLMs) on spatial reasoning.
  arXiv  Detail & Related papers  (2024-06-07T01:06:34Z)
• Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph [15.129079475322637]
  This work unveils the factual information an Large Language Models represents internally for sentence-level claim verification. We propose an end-to-end framework to decode factual knowledge embedded in token representations from a vector space to a set of ground predicates. Our framework employs activation patching, a vector-level technique that alters a token representation during inference, to extract encoded knowledge.
  arXiv  Detail & Related papers  (2024-04-04T17:45:59Z)
• Unveiling A Core Linguistic Region in Large Language Models [49.860260050718516]
  This paper conducts an analogical research using brain localization as a prototype. We have discovered a core region in large language models that corresponds to linguistic competence. We observe that an improvement in linguistic competence does not necessarily accompany an elevation in the model's knowledge level.
  arXiv  Detail & Related papers  (2023-10-23T13:31:32Z)
• GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding [45.36562604939258]
  This paper introduces GeoLM, a language model that enhances the understanding of geo-entities in natural language. We demonstrate that GeoLM exhibits promising capabilities in supporting toponym recognition, toponym linking, relation extraction, and geo-entity typing.
  arXiv  Detail & Related papers  (2023-10-23T01:20:01Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• Are Large Language Models Geospatially Knowledgeable? [21.401931052512595]
  This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within Large Language Models (LLM) With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models' geospatial reasoning capabilities.
  arXiv  Detail & Related papers  (2023-10-09T17:20:11Z)
• GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark [56.08664336835741]
  We propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks. We pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
  arXiv  Detail & Related papers  (2023-05-11T03:21:56Z)
• Geographic Adaptation of Pretrained Language Models [29.81557992080902]
  We introduce geoadaptation, an intermediate training step that couples language modeling with geolocation prediction in a multi-task learning setup. We show that the effectiveness of geoadaptation stems from its ability to geographically retrofit the representation space of the pretrained language models.
  arXiv  Detail & Related papers  (2022-03-16T11:55:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.