GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding

• URL: http://arxiv.org/abs/2310.14478v1
• Date: Mon, 23 Oct 2023 01:20:01 GMT
• Title: GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding
• Authors: Zekun Li, Wenxuan Zhou, Yao-Yi Chiang, Muhao Chen
• Abstract summary: This paper introduces GeoLM, a language model that enhances the understanding of geo-entities in natural language. We demonstrate that GeoLM exhibits promising capabilities in supporting toponym recognition, toponym linking, relation extraction, and geo-entity typing.
• Score: 45.36562604939258
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Humans subconsciously engage in geospatial reasoning when reading articles. We recognize place names and their spatial relations in text and mentally associate them with their physical locations on Earth. Although pretrained language models can mimic this cognitive process using linguistic context, they do not utilize valuable geospatial information in large, widely available geographical databases, e.g., OpenStreetMap. This paper introduces GeoLM, a geospatially grounded language model that enhances the understanding of geo-entities in natural language. GeoLM leverages geo-entity mentions as anchors to connect linguistic information in text corpora with geospatial information extracted from geographical databases. GeoLM connects the two types of context through contrastive learning and masked language modeling. It also incorporates a spatial coordinate embedding mechanism to encode distance and direction relations to capture geospatial context. In the experiment, we demonstrate that GeoLM exhibits promising capabilities in supporting toponym recognition, toponym linking, relation extraction, and geo-entity typing, which bridge the gap between natural language processing and geospatial sciences. The code is publicly available at https://github.com/knowledge-computing/geolm.

Related papers

• GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding [0.32885740436059047]
  GeoReasoner is a language model capable of reasoning on geospatially grounded natural language. It first leverages Large Language Models to generate a comprehensive location description based on linguistic inferences and distance information. It also encodes direction and distance information into spatial embedding via treating them as pseudo-sentences.
  arXiv  Detail & Related papers  (2024-08-21T06:35:21Z)
• Where on Earth Do Users Say They Are?: Geo-Entity Linking for Noisy Multilingual User Input [2.516307239032451]
  We present a method which represents real-world locations as averaged embeddings from labeled user-input location names. We show that our approach improves geo-entity linking on a global and multilingual social media dataset.
  arXiv  Detail & Related papers  (2024-04-29T15:18:33Z)
• GeoGalactica: A Scientific Large Language Model in Geoscience [95.15911521220052]
  Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP) We specialize an LLM into geoscience, by further pre-training the model with a vast amount of texts in geoscience, as well as supervised fine-tuning (SFT) the resulting model with our custom collected instruction tuning dataset. We train GeoGalactica over a geoscience-related text corpus containing 65 billion tokens, preserving as the largest geoscience-specific text corpus. Then we fine-tune the model with 1 million pairs of instruction-tuning
  arXiv  Detail & Related papers  (2023-12-31T09:22:54Z)
• Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching [60.645802236700035]
  Navigating drones through natural language commands remains challenging due to the dearth of accessible multi-modal datasets. We introduce GeoText-1652, a new natural language-guided geo-localization benchmark. This dataset is systematically constructed through an interactive human-computer process.
  arXiv  Detail & Related papers  (2023-11-21T17:52:30Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• Are Large Language Models Geospatially Knowledgeable? [21.401931052512595]
  This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within Large Language Models (LLM) With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models' geospatial reasoning capabilities.
  arXiv  Detail & Related papers  (2023-10-09T17:20:11Z)
• K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization [105.89544876731942]
  Large language models (LLMs) have achieved great success in general domains of natural language processing. We present the first-ever LLM in geoscience, K2, alongside a suite of resources developed to further promote LLM research within geoscience.
  arXiv  Detail & Related papers  (2023-06-08T09:29:05Z)
• GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark [56.08664336835741]
  We propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks. We pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
  arXiv  Detail & Related papers  (2023-05-11T03:21:56Z)
• SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation [25.52363878314735]
  SpaBERT provides a general-purpose geo-entity representation based on neighboring entities in geospatial data. SpaBERT is pretrained with masked language modeling and masked entity prediction tasks. We apply SpaBERT to two downstream tasks: geo-entity typing and geo-entity linking.
  arXiv  Detail & Related papers  (2022-10-21T19:42:32Z)
• Geographic Adaptation of Pretrained Language Models [29.81557992080902]
  We introduce geoadaptation, an intermediate training step that couples language modeling with geolocation prediction in a multi-task learning setup. We show that the effectiveness of geoadaptation stems from its ability to geographically retrofit the representation space of the pretrained language models.
  arXiv  Detail & Related papers  (2022-03-16T11:55:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.