Post-hurricane building damage assessment using street-view imagery and structured data: A multi-modal deep learning approach

• URL: http://arxiv.org/abs/2404.07399v1
• Date: Thu, 11 Apr 2024 00:23:28 GMT
• Title: Post-hurricane building damage assessment using street-view imagery and structured data: A multi-modal deep learning approach
• Authors: Zhuoqun Xue, Xiaojian Zhang, David O. Prevatt, Jennifer Bridge, Susu Xu, Xilei Zhao,
• Abstract summary: We propose a novel multi-modal approach for post-hurricane building damage classification, named the Multi-Modal Swin Transformer (MMST) We empirically train and evaluate the proposed MMST using data collected from the 2022 Hurricane Ian in Florida, USA. Results show that MMST outperforms all selected state-of-the-art benchmark models and can achieve an accuracy of 92.67%.
• Score: 1.748885212343545
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurately assessing building damage is critical for disaster response and recovery. However, many existing models for detecting building damage have poor prediction accuracy due to their limited capabilities of identifying detailed, comprehensive structural and/or non-structural damage from the street-view image. Additionally, these models mainly rely on the imagery data for damage classification, failing to account for other critical information, such as wind speed, building characteristics, evacuation zones, and distance of the building to the hurricane track. To address these limitations, in this study, we propose a novel multi-modal (i.e., imagery and structured data) approach for post-hurricane building damage classification, named the Multi-Modal Swin Transformer (MMST). We empirically train and evaluate the proposed MMST using data collected from the 2022 Hurricane Ian in Florida, USA. Results show that MMST outperforms all selected state-of-the-art benchmark models and can achieve an accuracy of 92.67%, which are 7.71% improvement in accuracy compared to Visual Geometry Group 16 (VGG-16). In addition to the street-view imagery data, building value, building age, and wind speed are the most important predictors for damage level classification. The proposed MMST can be deployed to assist in rapid damage assessment and guide reconnaissance efforts in future hurricanes.

Related papers

• DeepDamageNet: A two-step deep-learning model for multi-disaster building damage segmentation and classification using satellite imagery [12.869300064524122]
  We present a solution that performs the two most important tasks in building damage assessment, segmentation and classification, through deep-learning models. Our best model couples a building identification semantic segmentation convolutional neural network (CNN) to a building damage classification CNN, with a combined F1 score of 0.66. We find that though our model was able to identify buildings with relatively high accuracy, building damage classification across various disaster types is a difficult task.
  arXiv  Detail & Related papers  (2024-05-08T04:21:03Z)
• Causality-informed Rapid Post-hurricane Building Damage Detection in Large Scale from InSAR Imagery [6.331801334141028]
  Timely and accurate assessment of hurricane-induced building damage is crucial for effective post-hurricane response and recovery efforts. Recently, remote sensing technologies provide large-scale optical or Interferometric Synthetic Aperture Radar (InSAR) imagery data immediately after a disastrous event. These InSAR imageries often contain highly noisy and mixed signals induced by co-occurring or co-located building damage, flood, flood/wind-induced vegetation changes, as well as anthropogenic activities. This paper introduces an approach for rapid post-hurricane building damage detection from InSAR imagery.
  arXiv  Detail & Related papers  (2023-10-02T18:56:05Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• CarPatch: A Synthetic Benchmark for Radiance Field Evaluation on Vehicle Components [77.33782775860028]
  We introduce CarPatch, a novel synthetic benchmark of vehicles. In addition to a set of images annotated with their intrinsic and extrinsic camera parameters, the corresponding depth maps and semantic segmentation masks have been generated for each view. Global and part-based metrics have been defined and used to evaluate, compare, and better characterize some state-of-the-art techniques.
  arXiv  Detail & Related papers  (2023-07-24T11:59:07Z)
• Classification of structural building damage grades from multi-temporal photogrammetric point clouds using a machine learning model trained on virtual laser scanning data [58.720142291102135]
  We present a novel approach to automatically assess multi-class building damage from real-world point clouds. We use a machine learning model trained on virtual laser scanning (VLS) data. The model yields high multi-target classification accuracies (overall accuracy: 92.0% - 95.1%)
  arXiv  Detail & Related papers  (2023-02-24T12:04:46Z)
• Multi-view deep learning for reliable post-disaster damage classification [0.0]
  This study aims to enable more reliable automated post-disaster building damage classification using artificial intelligence (AI) and multi-view imagery. The proposed model is trained and validated on reconnaissance visual dataset containing expert-labeled, geotagged images of the inspected buildings following hurricane Harvey.
  arXiv  Detail & Related papers  (2022-08-06T01:04:13Z)
• Post-Hurricane Damage Assessment Using Satellite Imagery and Geolocation Features [0.2538209532048866]
  We propose a mixed data approach, which leverages publicly available satellite imagery and geolocation features of the affected area to identify damaged buildings after a hurricane. The method demonstrated significant improvement from performing a similar task using only imagery features, based on a case study of Hurricane Harvey affecting Greater Houston area in 2017. In this work, a creative choice of the geolocation features was made to provide extra information to the imagery features, but it is up to the users to decide which other features can be included to model the physical behavior of the events, depending on their domain knowledge and the type of disaster.
  arXiv  Detail & Related papers  (2020-12-15T21:30:19Z)
• Assessing out-of-domain generalization for robust building damage detection [78.6363825307044]
  Building damage detection can be automated by applying computer vision techniques to satellite imagery. Models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. We argue that future work should focus on the OOD regime instead.
  arXiv  Detail & Related papers  (2020-11-20T10:30:43Z)
• MSNet: A Multilevel Instance Segmentation Network for Natural Disaster Damage Assessment in Aerial Videos [74.22132693931145]
  We study the problem of efficiently assessing building damage after natural disasters like hurricanes, floods or fires. The first contribution is a new dataset, consisting of user-generated aerial videos from social media with annotations of instance-level building damage masks. The second contribution is a new model, namely MSNet, which contains novel region proposal network designs.
  arXiv  Detail & Related papers  (2020-06-30T02:23:05Z)
• RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery [83.49145695899388]
  RescueNet is a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-to-end. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods.
  arXiv  Detail & Related papers  (2020-04-15T19:52:09Z)
• An Attention-Based System for Damage Assessment Using Satellite Imagery [18.43310705820528]
  We present Siam-U-Net-Attn model - a multi-class deep learning model with an attention mechanism - to assess damage levels of buildings. We evaluate the proposed method on xView2, a large-scale building damage assessment dataset, and demonstrate that the proposed approach achieves accurate damage scale classification and building segmentation results simultaneously.
  arXiv  Detail & Related papers  (2020-04-14T16:37:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.