BIM-GPT: a Prompt-Based Virtual Assistant Framework for BIM Information Retrieval

• URL: http://arxiv.org/abs/2304.09333v1
• Date: Tue, 18 Apr 2023 22:46:02 GMT
• Title: BIM-GPT: a Prompt-Based Virtual Assistant Framework for BIM Information Retrieval
• Authors: Junwen Zheng, Martin Fischer
• Abstract summary: We introduce BIM-GPT, a prompt-based virtual assistant (VA) framework integrating BIM and generative pre-trained transformer (GPT) technologies to support NL-based IR. A prompt manager and dynamic template generate prompts for GPT models, enabling interpretation of NL queries, summarization of retrieved information, and answering BIM-related questions. In tests on a BIM IR dataset, our approach achieved 83.5% and 99.5% accuracy rates for classifying NL queries with no data and 2% data incorporated in prompts, respectively.
• Score: 0.8339831319589134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient information retrieval (IR) from building information models (BIMs) poses significant challenges due to the necessity for deep BIM knowledge or extensive engineering efforts for automation. We introduce BIM-GPT, a prompt-based virtual assistant (VA) framework integrating BIM and generative pre-trained transformer (GPT) technologies to support NL-based IR. A prompt manager and dynamic template generate prompts for GPT models, enabling interpretation of NL queries, summarization of retrieved information, and answering BIM-related questions. In tests on a BIM IR dataset, our approach achieved 83.5% and 99.5% accuracy rates for classifying NL queries with no data and 2% data incorporated in prompts, respectively. Additionally, we validated the functionality of BIM-GPT through a VA prototype for a hospital building. This research contributes to the development of effective and versatile VAs for BIM IR in the construction industry, significantly enhancing BIM accessibility and reducing engineering efforts and training data requirements for processing NL queries.

Related papers

• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation [72.70046559930555]
  We propose a generic RAG approach called Adaptive Note-Enhanced RAG (Adaptive-Note) for complex QA tasks. Specifically, Adaptive-Note introduces an overarching view of knowledge growth, iteratively gathering new information in the form of notes. In addition, we employ an adaptive, note-based stop-exploration strategy to decide "what to retrieve and when to stop" to encourage sufficient knowledge exploration.
  arXiv  Detail & Related papers  (2024-10-11T14:03:29Z)
• A Generalized LLM-Augmented BIM Framework: Application to a Speech-to-BIM system [0.0]
  The proposed framework consists of six steps: interpret-fill-match-structure-execute-check. The paper demonstrates the applicability of the proposed framework through implementing a speech-to- BIM application, NADIA-S.
  arXiv  Detail & Related papers  (2024-09-26T23:46:15Z)
• Towards commands recommender system in BIM authoring tool using transformers [0.7499722271664147]
  This study explores the potential of sequential recommendation systems to accelerate the BIM modeling process. By treating BIM software commands as recommendable items, we introduce a novel end-to-end approach that predicts the next-best command based on user historical interactions.
  arXiv  Detail & Related papers  (2024-06-02T17:47:06Z)
• Exploring the Capabilities of Large Multimodal Models on Dense Text [58.82262549456294]
  We propose the DT-VQA dataset, with 170k question-answer pairs. In this paper, we conduct a comprehensive evaluation of GPT4V, Gemini, and various open-source LMMs. We find that even with automatically labeled training datasets, significant improvements in model performance can be achieved.
  arXiv  Detail & Related papers  (2024-05-09T07:47:25Z)
• Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges [4.588192657854766]
  This survey focuses on approaches that apply pretrained transformer encoders like BERT to information retrieval (IR) We group them into six high-level categories: (i) handling long documents, (ii) integrating semantic information, (iii) balancing effectiveness and efficiency, (iv) predicting the weights of terms, (v) query expansion, and (vi) document expansion. We find that for specific tasks, finely tuned BERT encoders still outperform, and at a lower deployment cost.
  arXiv  Detail & Related papers  (2024-02-18T23:22:40Z)
• Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy [68.31760483418901]
  Large Language Models(LLMs) struggle with providing current information due to the outdated pre-training data. Existing methods for updating LLMs, such as knowledge editing and continual fine-tuning, have significant drawbacks in generalizability of new information. We identify the core challenge behind these drawbacks: the LM-logical discrepancy featuring the difference between language modeling probabilities and logical probabilities.
  arXiv  Detail & Related papers  (2023-05-29T19:48:37Z)
• Improving BIM Authoring Process Reproducibility with Enhanced BIM Logging [0.0]
  The authors developed the logger and reproducing algorithm using the Revit C# API. The enhanced BIM log was evaluated through a case study of Villa Savoye designed by Le Corbusier.
  arXiv  Detail & Related papers  (2023-05-29T11:52:23Z)
• Just Tell Me: Prompt Engineering in Business Process Management [63.08166397142146]
  GPT-3 and other language models (LMs) can effectively address various natural language processing (NLP) tasks. We argue that prompt engineering can help bring the capabilities of LMs to BPM research.
  arXiv  Detail & Related papers  (2023-04-14T14:55:19Z)
• An ontology-aided, natural language-based approach for multi-constraint BIM model querying [0.0]
  This paper presents a novel ontology-aided semantic to automatically map natural language queries (NLQs) that contain different constraints into computer-readable codes for querying complex BIM models. A case study about the design-checking of a real-world residential building demonstrates the practical value of the proposed approach in the construction industry.
  arXiv  Detail & Related papers  (2023-03-27T11:35:40Z)
• BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models [65.51390418485207]
  We propose a new approach of harvesting massive KGs of arbitrary relations from pretrained LMs. With minimal input of a relation definition, the approach efficiently searches in the vast entity pair space to extract diverse accurate knowledge. We deploy the approach to harvest KGs of over 400 new relations from different LMs.
  arXiv  Detail & Related papers  (2022-06-28T19:46:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.