AutoRepo: A general framework for multi-modal LLM-based automated construction reporting

• URL: http://arxiv.org/abs/2310.07944v2
• Date: Mon, 4 Dec 2023 18:13:15 GMT
• Title: AutoRepo: A general framework for multi-modal LLM-based automated construction reporting
• Authors: Hongxu Pu, Xincong Yang, Jing Li, Runhao Guo, Heng Li
• Abstract summary: This paper presents a novel framework named AutoRepo for automated generation of construction inspection reports. The framework was applied and tested on a real-world construction site, demonstrating its potential to expedite the inspection process.
• Score: 4.406834811182582
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring the safety, quality, and timely completion of construction projects is paramount, with construction inspections serving as a vital instrument towards these goals. Nevertheless, the predominantly manual approach of present-day inspections frequently results in inefficiencies and inadequate information management. Such methods often fall short of providing holistic, exhaustive assessments, consequently engendering regulatory oversights and potential safety hazards. To address this issue, this paper presents a novel framework named AutoRepo for automated generation of construction inspection reports. The unmanned vehicles efficiently perform construction inspections and collect scene information, while the multimodal large language models (LLMs) are leveraged to automatically generate the inspection reports. The framework was applied and tested on a real-world construction site, demonstrating its potential to expedite the inspection process, significantly reduce resource allocation, and produce high-quality, regulatory standard-compliant inspection reports. This research thus underscores the immense potential of multimodal large language models in revolutionizing construction inspection practices, signaling a significant leap forward towards a more efficient and safer construction management paradigm.

Related papers

• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Automatic Instantiation of Assurance Cases from Patterns Using Large Language Models [6.314768437420443]
  Large Language Models (LLMs) can generate assurance cases that comply with specific patterns. LLMs exhibit potential in the automatic generation of assurance cases, but their capabilities still fall short compared to human experts.
  arXiv  Detail & Related papers  (2024-10-07T20:58:29Z)
• Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles [6.836108615628114]
  We propose OmniTester: a framework that generates realistic and diverse scenarios within a simulation environment. In the experiments, we demonstrated the controllability and realism of our approaches in generating three types of challenging and complex scenarios.
  arXiv  Detail & Related papers  (2024-09-10T12:12:09Z)
• Claim Verification in the Age of Large Language Models: A Survey [37.32036088774565]
  We present a comprehensive account of recent claim verification frameworks using Large Language Models (LLMs) We describe the different components of the claim verification pipeline used in these frameworks in detail.
  arXiv  Detail & Related papers  (2024-08-26T14:45:03Z)
• Automatic benchmarking of large multimodal models via iterative experiment programming [71.78089106671581]
  We present APEx, the first framework for automatic benchmarking of LMMs. Given a research question expressed in natural language, APEx leverages a large language model (LLM) and a library of pre-specified tools to generate a set of experiments for the model at hand. The report drives the testing procedure: based on the current status of the investigation, APEx chooses which experiments to perform and whether the results are sufficient to draw conclusions.
  arXiv  Detail & Related papers  (2024-06-18T06:43:46Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations [76.19419888353586]
  Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations. We present our efforts to create and deploy a library of detectors: compact and easy-to-build classification models that provide labels for various harms.
  arXiv  Detail & Related papers  (2024-03-09T21:07:16Z)
• Computer Vision for Construction Progress Monitoring: A Real-Time Object Detection Approach [0.0]
  Construction progress monitoring (CPM) is essential for effective project management, ensuring on-time and on-budget delivery. Traditional CPM methods often rely on manual inspection and reporting, which are time-consuming and prone to errors. This paper proposes a novel approach for automated CPM using state-of-the-art object detection algorithms.
  arXiv  Detail & Related papers  (2023-05-24T12:27:42Z)
• Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models [75.75038268227554]
  Self-Checker is a framework comprising a set of plug-and-play modules that facilitate fact-checking. This framework provides a fast and efficient way to construct fact-checking systems in low-resource environments.
  arXiv  Detail & Related papers  (2023-05-24T01:46:07Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Construction material classification on imbalanced datasets for construction monitoring automation using Vision Transformer (ViT) architecture [0.0]
  The scope of automation in construction includes a wide range of stages, and monitoring construction projects is no exception. In this paper, a novel deep learning architecture is utilized, called Vision Transformer (ViT), for detecting and classifying construction materials. The achieved results revealed an accuracy of 100 percent in all parameters and also in each material category.
  arXiv  Detail & Related papers  (2021-08-21T15:29:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.