AutoRepo: A general framework for multi-modal LLM-based automated
construction reporting
- URL: http://arxiv.org/abs/2310.07944v2
- Date: Mon, 4 Dec 2023 18:13:15 GMT
- Title: AutoRepo: A general framework for multi-modal LLM-based automated
construction reporting
- Authors: Hongxu Pu, Xincong Yang, Jing Li, Runhao Guo, Heng Li
- Abstract summary: This paper presents a novel framework named AutoRepo for automated generation of construction inspection reports.
The framework was applied and tested on a real-world construction site, demonstrating its potential to expedite the inspection process.
- Score: 4.406834811182582
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Ensuring the safety, quality, and timely completion of construction projects
is paramount, with construction inspections serving as a vital instrument
towards these goals. Nevertheless, the predominantly manual approach of
present-day inspections frequently results in inefficiencies and inadequate
information management. Such methods often fall short of providing holistic,
exhaustive assessments, consequently engendering regulatory oversights and
potential safety hazards. To address this issue, this paper presents a novel
framework named AutoRepo for automated generation of construction inspection
reports. The unmanned vehicles efficiently perform construction inspections and
collect scene information, while the multimodal large language models (LLMs)
are leveraged to automatically generate the inspection reports. The framework
was applied and tested on a real-world construction site, demonstrating its
potential to expedite the inspection process, significantly reduce resource
allocation, and produce high-quality, regulatory standard-compliant inspection
reports. This research thus underscores the immense potential of multimodal
large language models in revolutionizing construction inspection practices,
signaling a significant leap forward towards a more efficient and safer
construction management paradigm.
Related papers
- Automatic benchmarking of large multimodal models via iterative experiment programming [71.78089106671581]
We present APEx, the first framework for automatic benchmarking of LMMs.
Given a research question expressed in natural language, APEx leverages a large language model (LLM) and a library of pre-specified tools to generate a set of experiments for the model at hand.
The report drives the testing procedure: based on the current status of the investigation, APEx chooses which experiments to perform and whether the results are sufficient to draw conclusions.
arXiv Detail & Related papers (2024-06-18T06:43:46Z) - AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios.
We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
arXiv Detail & Related papers (2024-03-26T04:27:56Z) - Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations [76.19419888353586]
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations.
We present our efforts to create and deploy a library of detectors: compact and easy-to-build classification models that provide labels for various harms.
arXiv Detail & Related papers (2024-03-09T21:07:16Z) - Automated Machine Learning in the smart construction era:Significance
and accessibility for industrial classification and regression tasks [6.206133097433925]
This paper explores the application of automated machine learning (AutoML) techniques to the construction industry.
By leveraging AutoML, construction professionals can now utilize software to process industrial data into ML models that assist in project management.
arXiv Detail & Related papers (2023-08-03T03:17:22Z) - Computer Vision for Construction Progress Monitoring: A Real-Time Object
Detection Approach [0.0]
Construction progress monitoring (CPM) is essential for effective project management, ensuring on-time and on-budget delivery.
Traditional CPM methods often rely on manual inspection and reporting, which are time-consuming and prone to errors.
This paper proposes a novel approach for automated CPM using state-of-the-art object detection algorithms.
arXiv Detail & Related papers (2023-05-24T12:27:42Z) - Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models [75.75038268227554]
Self-Checker is a framework comprising a set of plug-and-play modules that facilitate fact-checking.
This framework provides a fast and efficient way to construct fact-checking systems in low-resource environments.
arXiv Detail & Related papers (2023-05-24T01:46:07Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Reachability Verification Based Reliability Assessment for Deep
Reinforcement Learning Controlled Robotics and Autonomous Systems [17.679681019347065]
Deep Reinforcement Learning (DRL) has achieved impressive performance in robotics and autonomous systems (RAS)
A key challenge to its deployment in real-life operations is the presence of spuriously unsafe DRL policies.
This paper proposes a novel quantitative reliability assessment framework for DRL-controlled RAS.
arXiv Detail & Related papers (2022-10-26T19:25:46Z) - Construction material classification on imbalanced datasets for
construction monitoring automation using Vision Transformer (ViT)
architecture [0.0]
The scope of automation in construction includes a wide range of stages, and monitoring construction projects is no exception.
In this paper, a novel deep learning architecture is utilized, called Vision Transformer (ViT), for detecting and classifying construction materials.
The achieved results revealed an accuracy of 100 percent in all parameters and also in each material category.
arXiv Detail & Related papers (2021-08-21T15:29:56Z) - Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors.
Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf)
For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
arXiv Detail & Related papers (2021-04-16T21:35:08Z) - IFC models for (semi)automating common planning checks for building
permits [0.0]
A tool was developed to extract the necessary information from IFC models to check representative regulations.
While the case study is specific in location, regulations and input models, the type of issues encountered are a generally applicable example for automated code compliance checking.
arXiv Detail & Related papers (2020-11-03T15:29:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.