GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks

• URL: http://arxiv.org/abs/2410.17031v2
• Date: Wed, 23 Oct 2024 13:52:51 GMT
• Title: GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
• Authors: Shuyang Hou, Zhangxiao Shen, Anqi Zhao, Jianyuan Liang, Zhipeng Gui, Xuefeng Guan, Rui Li, Huayi Wu,
• Abstract summary: This paper presents and open-sources the GeoCode-PT and GeoCode-SFT corpora, along with the GeoCode-Eval evaluation dataset. By leveraging QRA and LoRA for pretraining and fine-tuning, we introduce GeoCode-GPT-7B, the first LLM focused on geospatial code generation. Experimental results show that GeoCode-GPT outperforms other models in multiple-choice accuracy by 9.1% to 32.1%, in code summarization ability by 5.4% to 21.7%, and in code generation capability by 1.2% to 25.1%.
• Score: 1.7687829461198472
• License:
• Abstract: The increasing demand for spatiotemporal data and modeling tasks in geosciences has made geospatial code generation technology a critical factor in enhancing productivity. Although large language models (LLMs) have demonstrated potential in code generation tasks, they often encounter issues such as refusal to code or hallucination in geospatial code generation due to a lack of domain-specific knowledge and code corpora. To address these challenges, this paper presents and open-sources the GeoCode-PT and GeoCode-SFT corpora, along with the GeoCode-Eval evaluation dataset. Additionally, by leveraging QLoRA and LoRA for pretraining and fine-tuning, we introduce GeoCode-GPT-7B, the first LLM focused on geospatial code generation, fine-tuned from Code Llama-7B. Furthermore, we establish a comprehensive geospatial code evaluation framework, incorporating option matching, expert validation, and prompt engineering scoring for LLMs, and systematically evaluate GeoCode-GPT-7B using the GeoCode-Eval dataset. Experimental results show that GeoCode-GPT outperforms other models in multiple-choice accuracy by 9.1% to 32.1%, in code summarization ability by 1.7% to 25.4%, and in code generation capability by 1.2% to 25.1%. This paper provides a solution and empirical validation for enhancing LLMs' performance in geospatial code generation, extends the boundaries of domain-specific model applications, and offers valuable insights into unlocking their potential in geospatial code generation.

Related papers

• CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models [91.15135237584771]
  Large language models (LLMs) can act as agents with capabilities to self-refine and improve generated code autonomously. We propose CodeTree, a framework for LLM agents to efficiently explore the search space in different stages of the code generation process. Specifically, we adopted a unified tree structure to explicitly explore different coding strategies, generate corresponding coding solutions, and subsequently refine the solutions.
  arXiv  Detail & Related papers  (2024-11-07T00:09:54Z)
• Geo-FuB: A Method for Constructing an Operator-Function Knowledge Base for Geospatial Code Generation Tasks Using Large Language Models [0.5242869847419834]
  This study introduces a framework to construct such a knowledge base, leveraging geospatial script semantics. An example knowledge base, Geo-FuB, built from 154,075 Google Earth Engine scripts, is available on GitHub.
  arXiv  Detail & Related papers  (2024-10-28T12:50:27Z)
• Can Large Language Models Generate Geospatial Code? [1.8449647922514607]
  This paper introduces GeoCode-Eval, a framework for assessing large language models' ability to generate geospatial code. We developed a benchmark dataset, GeoCode-Bench, consisting of 5,000 multiple-choice, 1,500 fill-in-the-blank, 1,500 true/false questions, and 1,000 subjective tasks. Using GeoCode-Bench, we evaluated three commercial closed-source LLMs, four open-source general-purpose LLMs, and 14 specialized code generation models.
  arXiv  Detail & Related papers  (2024-10-13T06:14:27Z)
• Evaluation of Code LLMs on Geospatial Code Generation [1.6834474847800562]
  Large Language Models (LLMs) can generate Python code for data science and machine learning applications. Here, we show how we constructed an evaluation benchmark for code generation models, based on a selection of geospatial tasks. Our dataset will hopefully contribute to the development new models capable of solving geospatial coding tasks with high accuracy.
  arXiv  Detail & Related papers  (2024-10-06T20:34:03Z)
• GeoGalactica: A Scientific Large Language Model in Geoscience [95.15911521220052]
  Large language models (LLMs) have achieved huge success for their general knowledge and ability to solve a wide spectrum of tasks in natural language processing (NLP) We specialize an LLM into geoscience, by further pre-training the model with a vast amount of texts in geoscience, as well as supervised fine-tuning (SFT) the resulting model with our custom collected instruction tuning dataset. We train GeoGalactica over a geoscience-related text corpus containing 65 billion tokens, preserving as the largest geoscience-specific text corpus. Then we fine-tune the model with 1 million pairs of instruction-tuning
  arXiv  Detail & Related papers  (2023-12-31T09:22:54Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking [61.60169764507917]
  Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates. We propose an innovative framework, namely Geo-Encoder, to more effectively integrate Chinese geographical semantics into re-ranking pipelines.
  arXiv  Detail & Related papers  (2023-09-04T13:44:50Z)
• GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPT [6.618846295332767]
  Decision-makers in GIS need to combine a series of spatial algorithms and operations to solve geospatial tasks. We develop a new framework called GeoGPT that can conduct geospatial data collection, processing, and analysis in an autonomous manner.
  arXiv  Detail & Related papers  (2023-07-16T03:03:59Z)
• K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization [105.89544876731942]
  Large language models (LLMs) have achieved great success in general domains of natural language processing. We present the first-ever LLM in geoscience, K2, alongside a suite of resources developed to further promote LLM research within geoscience.
  arXiv  Detail & Related papers  (2023-06-08T09:29:05Z)
• GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark [56.08664336835741]
  We propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks. We pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
  arXiv  Detail & Related papers  (2023-05-11T03:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.