A semi-supervised self-training method to develop assistive intelligence
for segmenting multiclass bridge elements from inspection videos
- URL: http://arxiv.org/abs/2109.05078v2
- Date: Tue, 14 Sep 2021 01:16:09 GMT
- Title: A semi-supervised self-training method to develop assistive intelligence
for segmenting multiclass bridge elements from inspection videos
- Authors: Muhammad Monjurul Karim, Ruwen Qin, Zhaozheng Yin, Genda Chen
- Abstract summary: This paper develops an assistive intelligence model for segmenting multiclass bridge elements from inspection videos.
A Mask Region-based Convolutional Neural Network (Mask R-CNN) pre-trained on a large public dataset was transferred to the new task.
A semi-supervised self-training (S$3$T) method was developed to engage experienced inspectors in refining the network.
- Score: 6.75013674088437
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Bridge inspection is an important step in preserving and rehabilitating
transportation infrastructure for extending their service lives. The
advancement of mobile robotic technology allows the rapid collection of a large
amount of inspection video data. However, the data are mainly images of complex
scenes, wherein a bridge of various structural elements mix with a cluttered
background. Assisting bridge inspectors in extracting structural elements of
bridges from the big complex video data, and sorting them out by classes, will
prepare inspectors for the element-wise inspection to determine the condition
of bridges. This paper is motivated to develop an assistive intelligence model
for segmenting multiclass bridge elements from inspection videos captured by an
aerial inspection platform. With a small initial training dataset labeled by
inspectors, a Mask Region-based Convolutional Neural Network (Mask R-CNN)
pre-trained on a large public dataset was transferred to the new task of
multiclass bridge element segmentation. Besides, the temporal coherence
analysis attempts to recover false negatives and identify the weakness that the
neural network can learn to improve. Furthermore, a semi-supervised
self-training (S$^3$T) method was developed to engage experienced inspectors in
refining the network iteratively. Quantitative and qualitative results from
evaluating the developed deep neural network demonstrate that the proposed
method can utilize a small amount of time and guidance from experienced
inspectors (3.58 hours for labeling 66 images) to build the network of
excellent performance (91.8% precision, 93.6% recall, and 92.7% f1-score).
Importantly, the paper illustrates an approach to leveraging the domain
knowledge and experiences of bridge professionals into computational
intelligence models to efficiently adapt the models to varied bridges in the
National Bridge Inventory.
Related papers
- Instance Segmentation of Reinforced Concrete Bridges with Synthetic Point Clouds [0.0]
National Bridge Inspection Standards require detailed element-level bridge inspections.
Traditionally, inspectors manually assign ratings by rating structural components based on damage.
We propose a novel approach for generating synthetic data using three distinct methods.
arXiv Detail & Related papers (2024-09-24T18:28:41Z) - BridgeData V2: A Dataset for Robot Learning at Scale [73.86688388408021]
BridgeData V2 is a large and diverse dataset of robotic manipulation behaviors.
It contains 60,096 trajectories collected across 24 environments on a publicly available low-cost robot.
arXiv Detail & Related papers (2023-08-24T17:41:20Z) - ALSO: Automotive Lidar Self-supervision by Occupancy estimation [70.70557577874155]
We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds.
The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled.
The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information.
arXiv Detail & Related papers (2022-12-12T13:10:19Z) - Label-Efficient Object Detection via Region Proposal Network
Pre-Training [58.50615557874024]
We propose a simple pretext task that provides an effective pre-training for the region proposal network (RPN)
In comparison with multi-stage detectors without RPN pre-training, our approach is able to consistently improve downstream task performance.
arXiv Detail & Related papers (2022-11-16T16:28:18Z) - A Multitask Deep Learning Model for Parsing Bridge Elements and
Segmenting Defect in Bridge Inspection Images [1.476043573732074]
The vast network of bridges in the United States raises a high requirement for its maintenance and rehabilitation.
The massive cost of visual inspection to assess the conditions of the bridges turns out to be a burden to some extent.
This paper develops a multitask deep neural network that fully utilizes such interdependence between bridge elements and defects.
arXiv Detail & Related papers (2022-09-06T02:48:15Z) - A Deep Neural Network for Multiclass Bridge Element Parsing in
Inspection Image Analysis [9.635496805334899]
This article aims to determine a suitable deep neural network (DNN) for parsing multiclass bridge elements in inspection images.
With data augmentation and a training sample of 130 images, a pre-trained HRNet is efficiently transferred to the task of structural element parsing.
arXiv Detail & Related papers (2022-09-05T21:02:08Z) - Proper Reuse of Image Classification Features Improves Object Detection [4.240984948137734]
A common practice in transfer learning is to initialize the downstream model weights by pre-training on a data-abundant upstream task.
Recent works show this is not strictly necessary under longer training regimes and provide recipes for training the backbone from scratch.
We show that an extreme form of knowledge preservation -- freezing the classifier-d backbone -- consistently improves many different detection models.
arXiv Detail & Related papers (2022-04-01T14:44:47Z) - HierMUD: Hierarchical Multi-task Unsupervised Domain Adaptation between
Bridges for Drive-by Damage Diagnosis [9.261126434781744]
We introduce a new framework that transfers the model learned from one bridge to diagnose damage in another bridge without any labels from the target bridge.
Our framework trains a hierarchical neural network model in an adversarial way to extract task-shared and task-specific features.
We evaluate our framework on experimental data collected from 2 bridges and 3 vehicles.
arXiv Detail & Related papers (2021-07-23T19:39:32Z) - Few-Cost Salient Object Detection with Adversarial-Paced Learning [95.0220555274653]
This paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only.
We name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario.
arXiv Detail & Related papers (2021-04-05T14:15:49Z) - TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training
Model [51.14840210957289]
Multi-object tracking is a fundamental vision problem that has been studied for a long time.
Despite the success of Tracking by Detection (TBD), this two-step method is too complicated to train in an end-to-end manner.
We propose a concise end-to-end model TubeTK which only needs one step training by introducing the bounding-tube" to indicate temporal-spatial locations of objects in a short video clip.
arXiv Detail & Related papers (2020-06-10T06:45:05Z) - End-to-End Object Detection with Transformers [88.06357745922716]
We present a new method that views object detection as a direct set prediction problem.
Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components.
The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss.
arXiv Detail & Related papers (2020-05-26T17:06:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.