Domain adaptation of large language models for geotechnical applications

• URL: http://arxiv.org/abs/2507.05613v2
• Date: Tue, 04 Nov 2025 01:18:57 GMT
• Title: Domain adaptation of large language models for geotechnical applications
• Authors: Lei Fan, Fangxue Liu, Cheng Chen,
• Abstract summary: Large language models (LLMs) are transforming opportunities in geotechnical engineering, where rely on complex, text-rich data.<n>This review critically examines four key adaptation strategies, including prompt engineering, retrieval augmented generation, domain-adaptive pretraining, and fine-tuning.<n>Findings show that domain-adapted LLMs substantially improve reasoning accuracy, automation, and interpretability, yet remain limited by data scarcity, validation challenges, and explainability concerns.
• Score: 5.576513036374959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid advancement of large language models (LLMs) is transforming opportunities in geotechnical engineering, where workflows rely on complex, text-rich data. While general-purpose LLMs demonstrate strong reasoning capabilities, their effectiveness in geotechnical applications is constrained by limited exposure to specialized terminology and domain logic. Thus, domain adaptation, tailoring general LLMs for geotechnical use, has become essential. This paper presents the first systematic review of LLM adaptation and application in geotechnical contexts. It critically examines four key adaptation strategies, including prompt engineering, retrieval augmented generation, domain-adaptive pretraining, and fine-tuning, and evaluates their comparative benefits, limitations, and implementation trends. This review synthesizes current applications spanning geological interpretation, subsurface characterization, design analysis, numerical modeling, risk assessment, and geotechnical education. Findings show that domain-adapted LLMs substantially improve reasoning accuracy, automation, and interpretability, yet remain limited by data scarcity, validation challenges, and explainability concerns. Future research directions are also suggested. This review establishes a critical foundation for developing geotechnically literate LLMs and guides researchers and practitioners in advancing the digital transformation of geotechnical engineering.

Related papers

• Speed Always Wins: A Survey on Efficient Architectures for Large Language Models [51.817121227562964]
  Large Language Models (LLMs) have delivered impressive results in language understanding, generation, reasoning, and pushes the ability boundary of multimodal models.<n> Transformer models, as the foundation of modern LLMs, offer a strong baseline with excellent scaling properties.<n>The traditional transformer architecture requires substantial computations and poses significant obstacles for large-scale training and practical deployment.
  arXiv  Detail & Related papers  (2025-08-13T14:13:46Z)
• Evaluating Large Language Models for Real-World Engineering Tasks [75.97299249823972]
  This paper introduces a curated database comprising over 100 questions derived from authentic, production-oriented engineering scenarios.<n>Using this dataset, we evaluate four state-of-the-art Large Language Models (LLMs)<n>Our results show that LLMs demonstrate strengths in basic temporal and structural reasoning but struggle significantly with abstract reasoning, formal modeling, and context-sensitive engineering logic.
  arXiv  Detail & Related papers  (2025-05-12T14:05:23Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• A Survey on Post-training of Large Language Models [185.51013463503946]
  Large Language Models (LLMs) have fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific exploration.<n>These challenges necessitate advanced post-training language models (PoLMs) to address shortcomings, such as restricted reasoning capacities, ethical uncertainties, and suboptimal domain-specific performance.<n>This paper presents the first comprehensive survey of PoLMs, systematically tracing their evolution across five core paradigms: Fine-tuning, which enhances task-specific accuracy; Alignment, which ensures ethical coherence and alignment with human preferences; Reasoning, which advances multi-step inference despite challenges in reward design; Integration and Adaptation, which
  arXiv  Detail & Related papers  (2025-03-08T05:41:42Z)
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models [131.10969986056]
  Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications.<n>Post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations.
  arXiv  Detail & Related papers  (2025-02-28T18:59:54Z)
• Enhancing Reasoning to Adapt Large Language Models for Domain-Specific Applications [4.122613733775677]
  SOLOMON is a novel Neuro-inspired Large Language Model (LLM) Reasoning Network architecture.<n>We show how SOLOMON enables swift adaptation of general-purpose LLMs to specialized tasks by leveraging Prompt Engineering and In-Context Learning techniques.<n>Results show that SOLOMON instances significantly outperform their baseline LLM counterparts and achieve performance comparable to state-of-the-art reasoning model, o1-preview.
  arXiv  Detail & Related papers  (2025-02-05T19:27:24Z)
• PEACE: Empowering Geologic Map Holistic Understanding with MLLMs [64.58959634712215]
  Geologic map, as a fundamental diagram in geology science, provides critical insights into the structure and composition of Earth's subsurface and surface.<n>Despite their significance, current Multimodal Large Language Models (MLLMs) often fall short in geologic map understanding.<n>To quantify this gap, we construct GeoMap-Bench, the first-ever benchmark for evaluating MLLMs in geologic map understanding.
  arXiv  Detail & Related papers  (2025-01-10T18:59:42Z)
• Self-Supervised Representation Learning for Geospatial Objects: A Survey [21.504978593542354]
  Self-supervised learning (SSL) has garnered increasing attention for its ability to learn effective and generalizable representations directly from data without extensive labeled supervision.<n>This paper presents a survey of SSL techniques specifically applied to or developed for geospatial objects in three primary geometric vector types: Point, Polyline, and Polygon.<n>We examine the emerging trends in SSL for geospatial objects, particularly the gradual advancements towards geospatial foundation models.
  arXiv  Detail & Related papers  (2024-08-22T05:28:22Z)
• Future-proofing geotechnics workflows: accelerating problem-solving with large language models [2.8414492326907577]
  This paper delves into the innovative application of Large Language Models in geotechnical engineering, as explored in a hands-on workshop held in Tokyo, Japan. The paper discusses the potential of LLMs to transform geotechnical engineering practices, highlighting their proficiency in handling a range of tasks from basic data analysis to complex problem-solving.
  arXiv  Detail & Related papers  (2023-12-14T05:17:27Z)
• The Efficiency Spectrum of Large Language Models: An Algorithmic Survey [54.19942426544731]
  The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains. This paper examines the multi-faceted dimensions of efficiency essential for the end-to-end algorithmic development of LLMs.
  arXiv  Detail & Related papers  (2023-12-01T16:00:25Z)
• Challenges in data-based geospatial modeling for environmental research and practice [19.316860936437823]
  Data-based geospatial modelling using machine learning (ML) has gained popularity in environmental research. This survey reviews common nuances in geospatial modelling, such as imbalanced data, spatial autocorrelation, prediction errors, model generalisation, domain specificity, and uncertainty estimation.
  arXiv  Detail & Related papers  (2023-11-18T12:30:49Z)
• Are Large Language Models Geospatially Knowledgeable? [21.401931052512595]
  This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within Large Language Models (LLM) With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models' geospatial reasoning capabilities.
  arXiv  Detail & Related papers  (2023-10-09T17:20:11Z)
• K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization [105.89544876731942]
  Large language models (LLMs) have achieved great success in general domains of natural language processing. We present the first-ever LLM in geoscience, K2, alongside a suite of resources developed to further promote LLM research within geoscience.
  arXiv  Detail & Related papers  (2023-06-08T09:29:05Z)
• Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey [100.24095818099522]
  Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) They provide a highly useful, task-agnostic foundation for a wide range of applications. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles.
  arXiv  Detail & Related papers  (2023-05-30T03:00:30Z)
• Geotechnical Parrot Tales (GPT): Harnessing Large Language Models in geotechnical engineering [2.132096006921048]
  GPT models can generate plausible-sounding but false outputs, leading to hallucinations. By integrating GPT into geotechnical engineering, professionals can streamline their work and develop sustainable and resilient infrastructure systems.
  arXiv  Detail & Related papers  (2023-04-04T21:47:41Z)
• Information Extraction in Low-Resource Scenarios: Survey and Perspective [56.5556523013924]
  Information Extraction seeks to derive structured information from unstructured texts. This paper presents a review of neural approaches to low-resource IE from emphtraditional and emphLLM-based perspectives.
  arXiv  Detail & Related papers  (2022-02-16T13:44:00Z)
• Applications of physics-informed scientific machine learning in subsurface science: A survey [64.0476282000118]
  Geosystems are geological formations altered by humans activities such as fossil energy exploration, waste disposal, geologic carbon sequestration, and renewable energy generation. The responsible use and exploration of geosystems are thus critical to the geosystem governance, which in turn depends on the efficient monitoring, risk assessment, and decision support tools for practical implementation. Fast advances in machine learning algorithms and novel sensing technologies in recent years have presented new opportunities for the subsurface research community to improve the efficacy and transparency of geosystem governance.
  arXiv  Detail & Related papers  (2021-04-10T13:40:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.