A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

• URL: http://arxiv.org/abs/2209.02190v1
• Date: Tue, 6 Sep 2022 02:48:15 GMT
• Title: A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images
• Authors: Chenyu Zhang, Muhammad Monjurul Karim, Ruwen Qin
• Abstract summary: The vast network of bridges in the United States raises a high requirement for its maintenance and rehabilitation. The massive cost of visual inspection to assess the conditions of the bridges turns out to be a burden to some extent. This paper develops a multitask deep neural network that fully utilizes such interdependence between bridge elements and defects.
• Score: 1.476043573732074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The vast network of bridges in the United States raises a high requirement for its maintenance and rehabilitation. The massive cost of manual visual inspection to assess the conditions of the bridges turns out to be a burden to some extent. Advanced robots have been leveraged to automate inspection data collection. Automating the segmentations of multiclass elements, as well as surface defects on the elements, in the large volume of inspection image data would facilitate an efficient and effective assessment of the bridge condition. Training separate single-task networks for element parsing (i.e., semantic segmentation of multiclass elements) and defect segmentation fails to incorporate the close connection between these two tasks in the inspection images where both recognizable structural elements and apparent surface defects are present. This paper is motivated to develop a multitask deep neural network that fully utilizes such interdependence between bridge elements and defects to boost the performance and generalization of the model. Furthermore, the effectiveness of the proposed network designs in improving the task performance was investigated, including feature decomposition, cross-talk sharing, and multi-objective loss function. A dataset with pixel-level labels of bridge elements and corrosion was developed for training and assessment of the models. Quantitative and qualitative results from evaluating the developed multitask deep neural network demonstrate that the recommended network outperforms the independent single-task networks not only in performance (2.59% higher mIoU on bridge parsing and 1.65% on corrosion segmentation) but also in computational time and implementation capability.

Related papers

• Renormalized Connection for Scale-preferred Object Detection in Satellite Imagery [51.83786195178233]
  We design a Knowledge Discovery Network (KDN) to implement the renormalization group theory in terms of efficient feature extraction. Renormalized connection (RC) on the KDN enables synergistic focusing'' of multi-scale features. RCs extend the multi-level feature's divide-and-conquer'' mechanism of the FPN-based detectors to a wide range of scale-preferred tasks.
  arXiv  Detail & Related papers  (2024-09-09T13:56:22Z)
• FriendNet: Detection-Friendly Dehazing Network [24.372610892854283]
  We propose an effective architecture that bridges image dehazing and object detection together via guidance information and task-driven learning. FriendNet aims to deliver both high-quality perception and high detection capacity.
  arXiv  Detail & Related papers  (2024-03-07T12:19:04Z)
• Evaluation of Data Augmentation and Loss Functions in Semantic Image Segmentation for Drilling Tool Wear Detection [0.0]
  We present a U-Net based semantic image segmentation pipeline, deployed on microscopy images of cutting inserts. The wear area is differentiated in two different types, resulting in a multiclass classification problem. We find, that the best performing models are binary models, trained on data with moderate augmentation and an IoU-based loss function.
  arXiv  Detail & Related papers  (2023-02-10T14:21:41Z)
• A Deep Neural Network for Multiclass Bridge Element Parsing in Inspection Image Analysis [9.635496805334899]
  This article aims to determine a suitable deep neural network (DNN) for parsing multiclass bridge elements in inspection images. With data augmentation and a training sample of 130 images, a pre-trained HRNet is efficiently transferred to the task of structural element parsing.
  arXiv  Detail & Related papers  (2022-09-05T21:02:08Z)
• A semi-supervised self-training method to develop assistive intelligence for segmenting multiclass bridge elements from inspection videos [6.75013674088437]
  This paper develops an assistive intelligence model for segmenting multiclass bridge elements from inspection videos. A Mask Region-based Convolutional Neural Network (Mask R-CNN) pre-trained on a large public dataset was transferred to the new task. A semi-supervised self-training (S$3$T) method was developed to engage experienced inspectors in refining the network.
  arXiv  Detail & Related papers  (2021-09-10T19:22:45Z)
• Tasks Integrated Networks: Joint Detection and Retrieval for Image Search [99.49021025124405]
  In many real-world searching scenarios (e.g., video surveillance), the objects are seldom accurately detected or annotated. We first introduce an end-to-end Integrated Net (I-Net), which has three merits. We further propose an improved I-Net, called DC-I-Net, which makes two new contributions.
  arXiv  Detail & Related papers  (2020-09-03T03:57:50Z)
• Anchor-free Small-scale Multispectral Pedestrian Detection [88.7497134369344]
  We propose a method for effective and efficient multispectral fusion of the two modalities in an adapted single-stage anchor-free base architecture. We aim at learning pedestrian representations based on object center and scale rather than direct bounding box predictions. Results show our method's effectiveness in detecting small-scaled pedestrians.
  arXiv  Detail & Related papers  (2020-08-19T13:13:01Z)
• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference [75.95287293847697]
  Two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning) Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
  arXiv  Detail & Related papers  (2020-07-24T14:44:46Z)
• One-Shot Object Detection without Fine-Tuning [62.39210447209698]
  We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module. We also propose novel training strategies that effectively improve detection performance. Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
  arXiv  Detail & Related papers  (2020-05-08T01:59:23Z)
• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking [92.48078680697311]
  Multi-object tracking (MOT) is an important problem in computer vision. We present a simple yet effective approach termed as FairMOT based on the anchor-free object detection architecture CenterNet. The approach achieves high accuracy for both detection and tracking.
  arXiv  Detail & Related papers  (2020-04-04T08:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.