SAM-based instance segmentation models for the automation of structural damage detection

• URL: http://arxiv.org/abs/2401.15266v2
• Date: Tue, 30 Jan 2024 14:11:07 GMT
• Title: SAM-based instance segmentation models for the automation of structural damage detection
• Authors: Zehao Ye, Lucy Lovell, Asaad Faramarzi and Jelena Ninic
• Abstract summary: We present a data set for instance segmentation with 1,300 annotated images (640 pixels x 640 pixels), named as M1300, covering bricks, broken bricks, and cracks. We test several leading algorithms for benchmarking, including the latest large-scale model, the prompt-based Segment Anything Model (SAM) We propose two novel methods for automation of SAM execution. The first method involves abandoning the prompt encoder and connecting the SAM encoder to other decoders, while the second method introduces a learnable self-generating prompter.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automating visual inspection for capturing defects based on civil structures appearance is crucial due to its currently labour-intensive and time-consuming nature. An important aspect of automated inspection is image acquisition, which is rapid and cost-effective considering the pervasive developments in both software and hardware computing in recent years. Previous studies largely focused on concrete and asphalt, with less attention to masonry cracks. The latter also lacks publicly available datasets. In this paper, we first present a corresponding data set for instance segmentation with 1,300 annotated images (640 pixels x 640 pixels), named as MCrack1300, covering bricks, broken bricks, and cracks. We then test several leading algorithms for benchmarking, including the latest large-scale model, the prompt-based Segment Anything Model (SAM). We fine-tune the encoder using Low-Rank Adaptation (LoRA) and proposed two novel methods for automation of SAM execution. The first method involves abandoning the prompt encoder and connecting the SAM encoder to other decoders, while the second method introduces a learnable self-generating prompter. In order to ensure the seamless integration of the two proposed methods with SAM encoder section, we redesign the feature extractor. Both proposed methods exceed state-of-the-art performance, surpassing the best benchmark by approximately 3% for all classes and around 6% for cracks specifically. Based on successful detection, we propose a method based on a monocular camera and the Hough Line Transform to automatically transform images into orthographic projection maps. By incorporating known real sizes of brick units, we accurately estimate crack dimensions, with the results differing by less than 10% from those obtained by laser scanning. Overall, we address important research gaps in automated masonry crack detection and size estimation.

Related papers

• Unsupervised Fault Detection using SAM with a Moving Window Approach [0.0]
  We present an unsupervised method that uses the high end Segment Anything Model (SAM) and a moving window approach. We aim to overcome these challenges without requiring fine tun ing or labeled data.
  arXiv  Detail & Related papers  (2024-07-08T18:12:29Z)
• A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM) [0.0]
  This paper outlines a method we call the Multispectral Automated Transfer Technique (MATT) By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models.
  arXiv  Detail & Related papers  (2024-02-18T01:01:13Z)
• Small Effect Sizes in Malware Detection? Make Harder Train/Test Splits! [51.668411293817464]
  Industry practitioners care about small improvements in malware detection accuracy because their models are deployed to hundreds of millions of machines. Academic research is often restrained to public datasets on the order of ten thousand samples. We devise an approach to generate a benchmark of difficulty from a pool of available samples.
  arXiv  Detail & Related papers  (2023-12-25T21:25:55Z)
• Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors [117.61449210940955]
  We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level. We introduce an approach to weight tokens based on motion gradients, thus shifting the focus from the static background scene to the foreground objects. We generate synthetic abnormal events to augment the training videos, and task the masked AE model to jointly reconstruct the original frames.
  arXiv  Detail & Related papers  (2023-06-21T06:18:05Z)
• Read Pointer Meters in complex environments based on a Human-like Alignment and Recognition Algorithm [16.823681016882315]
  We propose a human-like alignment and recognition algorithm to overcome these problems. A Spatial Transformed Module(STM) is proposed to obtain the front view of images in a self-autonomous way. A Value Acquisition Module(VAM) is proposed to infer accurate meter values by an end-to-end trained framework.
  arXiv  Detail & Related papers  (2023-02-28T05:37:04Z)
• Cut and Learn for Unsupervised Object Detection and Instance Segmentation [65.43627672225624]
  Cut-and-LEaRn (CutLER) is a simple approach for training unsupervised object detection and segmentation models. CutLER is a zero-shot unsupervised detector and improves detection performance AP50 by over 2.7 times on 11 benchmarks.
  arXiv  Detail & Related papers  (2023-01-26T18:57:13Z)
• Self-Supervised Clustering on Image-Subtracted Data with Deep-Embedded Self-Organizing Map [0.0]
  Self-supervised machine learning model, the deep-embedded self-organizing map (DESOM) is applied to real-bogus classification problem. We demonstrate different model training approaches, and find that our best DESOM classifier shows a missed detection rate of 6.6% with a false positive rate of 1.5%.
  arXiv  Detail & Related papers  (2022-09-14T02:37:06Z)
• Exploring Resolution and Degradation Clues as Self-supervised Signal for Low Quality Object Detection [77.3530907443279]
  We propose a novel self-supervised framework to detect objects in degraded low resolution images. Our methods has achieved superior performance compared with existing methods when facing variant degradation situations.
  arXiv  Detail & Related papers  (2022-08-05T09:36:13Z)
• An Attention-based Method for Action Unit Detection at the 3rd ABAW Competition [6.229820412732652]
  This paper describes our submission to the third Affective Behavior Analysis in-the-wild (ABAW) competition 2022. We proposed a method for detecting facial action units in the video. We achieved a macro F1 score of 0.48 on the ABAW challenge validation set compared to 0.39 from the baseline model.
  arXiv  Detail & Related papers  (2022-03-23T14:07:39Z)
• When Liebig's Barrel Meets Facial Landmark Detection: A Practical Model [87.25037167380522]
  We propose a model that is accurate, robust, efficient, generalizable, and end-to-end trainable. In order to achieve a better accuracy, we propose two lightweight modules. DQInit dynamically initializes the queries of decoder from the inputs, enabling the model to achieve as good accuracy as the ones with multiple decoder layers. QAMem is designed to enhance the discriminative ability of queries on low-resolution feature maps by assigning separate memory values to each query rather than a shared one.
  arXiv  Detail & Related papers  (2021-05-27T13:51:42Z)
• SADet: Learning An Efficient and Accurate Pedestrian Detector [68.66857832440897]
  This paper proposes a series of systematic optimization strategies for the detection pipeline of one-stage detector. It forms a single shot anchor-based detector (SADet) for efficient and accurate pedestrian detection. Though structurally simple, it presents state-of-the-art result and real-time speed of $20$ FPS for VGA-resolution images.
  arXiv  Detail & Related papers  (2020-07-26T12:32:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.