Domain adaptation of large language models for geotechnical applications

• URL: http://arxiv.org/abs/2507.05613v1
• Date: Tue, 08 Jul 2025 02:45:44 GMT
• Title: Domain adaptation of large language models for geotechnical applications
• Authors: Lei Fan, Fangxue Liu, Cheng Chen,
• Abstract summary: This paper presents the first survey of the adaptation and application of large language models (LLMs) in geotechnical engineering.<n>It outlines key methodologies for adaptation to geotechnical domain, including prompt engineering, retrieval-augmented generation, domain-adaptive pretraining, and fine-tuning.<n>The survey examines the state-of-the-art applications of geotechnical-adapted LLMs, including geological interpretation, subsurface characterization, site planning, design calculations, numerical modeling, safety and risk assessment, and educational tutoring.
• Score: 3.839199344030664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent developments in large language models (LLMs) are opening up new opportunities in geotechnical engineering and engineering geology. While general-purpose LLMs possess broad capabilities, effective application in geotechnics often requires domain-specific adaptation. Such tailored LLMs are increasingly employed to streamline geotechnical workflows. This paper presents the first survey of the adaptation and application of LLMs in geotechnical engineering. It outlines key methodologies for adaptation to geotechnical domain, including prompt engineering, retrieval-augmented generation, domain-adaptive pretraining, and fine-tuning. The survey examines the state-of-the-art applications of geotechnical-adapted LLMs, including geological interpretation, subsurface characterization, site planning, design calculations, numerical modeling, safety and risk assessment, and educational tutoring. It also analyzes benefits and limitations of geotechnical-adapted LLMs, and identifies promising directions for future research in this interdisciplinary discipline. The findings serve as a valuable resource for practitioners seeking to integrate LLMs into geotechnical practice, while also providing a foundation to stimulate further investigation within the academic community.

Related papers

• Speed Always Wins: A Survey on Efficient Architectures for Large Language Models [51.817121227562964]
  Large Language Models (LLMs) have delivered impressive results in language understanding, generation, reasoning, and pushes the ability boundary of multimodal models.<n> Transformer models, as the foundation of modern LLMs, offer a strong baseline with excellent scaling properties.<n>The traditional transformer architecture requires substantial computations and poses significant obstacles for large-scale training and practical deployment.
  arXiv  Detail & Related papers  (2025-08-13T14:13:46Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• A Survey on Post-training of Large Language Models [185.51013463503946]
  Large Language Models (LLMs) have fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific exploration.<n>These challenges necessitate advanced post-training language models (PoLMs) to address shortcomings, such as restricted reasoning capacities, ethical uncertainties, and suboptimal domain-specific performance.<n>This paper presents the first comprehensive survey of PoLMs, systematically tracing their evolution across five core paradigms: Fine-tuning, which enhances task-specific accuracy; Alignment, which ensures ethical coherence and alignment with human preferences; Reasoning, which advances multi-step inference despite challenges in reward design; Integration and Adaptation, which
  arXiv  Detail & Related papers  (2025-03-08T05:41:42Z)
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models [131.10969986056]
  Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications.<n>Post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations.
  arXiv  Detail & Related papers  (2025-02-28T18:59:54Z)
• Enhancing Reasoning to Adapt Large Language Models for Domain-Specific Applications [4.122613733775677]
  SOLOMON is a novel Neuro-inspired Large Language Model (LLM) Reasoning Network architecture.<n>We show how SOLOMON enables swift adaptation of general-purpose LLMs to specialized tasks by leveraging Prompt Engineering and In-Context Learning techniques.<n>Results show that SOLOMON instances significantly outperform their baseline LLM counterparts and achieve performance comparable to state-of-the-art reasoning model, o1-preview.
  arXiv  Detail & Related papers  (2025-02-05T19:27:24Z)
• PEACE: Empowering Geologic Map Holistic Understanding with MLLMs [64.58959634712215]
  Geologic map, as a fundamental diagram in geology science, provides critical insights into the structure and composition of Earth's subsurface and surface.<n>Despite their significance, current Multimodal Large Language Models (MLLMs) often fall short in geologic map understanding.<n>To quantify this gap, we construct GeoMap-Bench, the first-ever benchmark for evaluating MLLMs in geologic map understanding.
  arXiv  Detail & Related papers  (2025-01-10T18:59:42Z)
• Self-Supervised Representation Learning for Geospatial Objects: A Survey [21.504978593542354]
  Self-supervised learning (SSL) has garnered increasing attention for its ability to learn effective and generalizable representations directly from data without extensive labeled supervision.<n>This paper presents a survey of SSL techniques specifically applied to or developed for geospatial objects in three primary geometric vector types: Point, Polyline, and Polygon.<n>We examine the emerging trends in SSL for geospatial objects, particularly the gradual advancements towards geospatial foundation models.
  arXiv  Detail & Related papers  (2024-08-22T05:28:22Z)
• Future-proofing geotechnics workflows: accelerating problem-solving with large language models [2.8414492326907577]
  This paper delves into the innovative application of Large Language Models in geotechnical engineering, as explored in a hands-on workshop held in Tokyo, Japan. The paper discusses the potential of LLMs to transform geotechnical engineering practices, highlighting their proficiency in handling a range of tasks from basic data analysis to complex problem-solving.
  arXiv  Detail & Related papers  (2023-12-14T05:17:27Z)
• The Efficiency Spectrum of Large Language Models: An Algorithmic Survey [54.19942426544731]
  The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains. This paper examines the multi-faceted dimensions of efficiency essential for the end-to-end algorithmic development of LLMs.
  arXiv  Detail & Related papers  (2023-12-01T16:00:25Z)
• Challenges in data-based geospatial modeling for environmental research and practice [19.316860936437823]
  Data-based geospatial modelling using machine learning (ML) has gained popularity in environmental research. This survey reviews common nuances in geospatial modelling, such as imbalanced data, spatial autocorrelation, prediction errors, model generalisation, domain specificity, and uncertainty estimation.
  arXiv  Detail & Related papers  (2023-11-18T12:30:49Z)
• Are Large Language Models Geospatially Knowledgeable? [21.401931052512595]
  This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within Large Language Models (LLM) With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models' geospatial reasoning capabilities.
  arXiv  Detail & Related papers  (2023-10-09T17:20:11Z)
• K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization [105.89544876731942]
  Large language models (LLMs) have achieved great success in general domains of natural language processing. We present the first-ever LLM in geoscience, K2, alongside a suite of resources developed to further promote LLM research within geoscience.
  arXiv  Detail & Related papers  (2023-06-08T09:29:05Z)
• Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey [100.24095818099522]
  Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) They provide a highly useful, task-agnostic foundation for a wide range of applications. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles.
  arXiv  Detail & Related papers  (2023-05-30T03:00:30Z)
• Geotechnical Parrot Tales (GPT): Harnessing Large Language Models in geotechnical engineering [2.132096006921048]
  GPT models can generate plausible-sounding but false outputs, leading to hallucinations. By integrating GPT into geotechnical engineering, professionals can streamline their work and develop sustainable and resilient infrastructure systems.
  arXiv  Detail & Related papers  (2023-04-04T21:47:41Z)
• Information Extraction in Low-Resource Scenarios: Survey and Perspective [56.5556523013924]
  Information Extraction seeks to derive structured information from unstructured texts. This paper presents a review of neural approaches to low-resource IE from emphtraditional and emphLLM-based perspectives.
  arXiv  Detail & Related papers  (2022-02-16T13:44:00Z)
• Applications of physics-informed scientific machine learning in subsurface science: A survey [64.0476282000118]
  Geosystems are geological formations altered by humans activities such as fossil energy exploration, waste disposal, geologic carbon sequestration, and renewable energy generation. The responsible use and exploration of geosystems are thus critical to the geosystem governance, which in turn depends on the efficient monitoring, risk assessment, and decision support tools for practical implementation. Fast advances in machine learning algorithms and novel sensing technologies in recent years have presented new opportunities for the subsurface research community to improve the efficacy and transparency of geosystem governance.
  arXiv  Detail & Related papers  (2021-04-10T13:40:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.