Post-Hurricane Debris Segmentation Using Fine-Tuned Foundational Vision Models

• URL: http://arxiv.org/abs/2504.12542v2
• Date: Sat, 19 Apr 2025 01:05:36 GMT
• Title: Post-Hurricane Debris Segmentation Using Fine-Tuned Foundational Vision Models
• Authors: Kooshan Amini, Yuhao Liu, Jamie Ellen Padgett, Guha Balakrishnan, Ashok Veeraraghavan,
• Abstract summary: This work introduces an open-source dataset comprising approximately 1,200 manually annotated aerial RGB images from Hurricanes Ian, Ida, and Ike.<n>To mitigate human biases and enhance data quality, labels from multiple annotators are strategically aggregated and visual prompt engineering is employed.<n>The resulting fine-tuned model, named fCLIPSeg, achieves a Dice score of 0.70 on data from Hurricane Ida with virtually no false positives in debris-free areas.
• Score: 18.008592164636664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Timely and accurate detection of hurricane debris is critical for effective disaster response and community resilience. While post-disaster aerial imagery is readily available, robust debris segmentation solutions applicable across multiple disaster regions remain limited. Developing a generalized solution is challenging due to varying environmental and imaging conditions that alter debris' visual signatures across different regions, further compounded by the scarcity of training data. This study addresses these challenges by fine-tuning pre-trained foundational vision models, achieving robust performance with a relatively small, high-quality dataset. Specifically, this work introduces an open-source dataset comprising approximately 1,200 manually annotated aerial RGB images from Hurricanes Ian, Ida, and Ike. To mitigate human biases and enhance data quality, labels from multiple annotators are strategically aggregated and visual prompt engineering is employed. The resulting fine-tuned model, named fCLIPSeg, achieves a Dice score of 0.70 on data from Hurricane Ida -- a disaster event entirely excluded during training -- with virtually no false positives in debris-free areas. This work presents the first event-agnostic debris segmentation model requiring only standard RGB imagery during deployment, making it well-suited for rapid, large-scale post-disaster impact assessments and recovery planning.

Related papers

• Pseudo-Label Guided Real-World Image De-weathering: A Learning Framework with Imperfect Supervision [57.5699142476311]
  We propose a unified solution for real-world image de-weathering with non-ideal supervision.<n>Our method exhibits significant advantages when trained on imperfectly aligned de-weathering datasets.
  arXiv  Detail & Related papers  (2025-04-14T07:24:03Z)
• Deep Self-Supervised Disturbance Mapping with the OPERA Sentinel-1 Radiometric Terrain Corrected SAR Backscatter Product [41.94295877935867]
  Mapping land surface disturbances supports disaster response, resource and ecosystem management, and climate adaptation efforts.<n> Synthetic aperture radar (SAR) is an invaluable tool for disturbance mapping, providing consistent time-series images of the ground regardless of weather or illumination conditions.<n>NASA's Observational Products for End-Users from Remote Sensing Analysis (OPERA) project released the near-global Radiometric Terrain Corrected SAR backscatter from Sentinel-1 (RTC-S1) dataset in October 2023.<n>In this work, we utilize this new dataset to systematically analyze land surface disturbances.
  arXiv  Detail & Related papers  (2025-01-15T20:24:18Z)
• Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model [17.016411785224317]
  We introduce DAVI (Disaster Assessment with VIsion foundation model), a novel approach that addresses domain disparities and detects structural damage at the building level without requiring ground-truth labels for target regions.<n>DAVI combines task-specific knowledge from a model trained on source regions with task-agnostic knowledge from an image segmentation model to generate pseudo labels indicating potential damage in target regions.<n>It then utilizes a two-stage refinement process, which operate at both pixel and image levels, to accurately identify changes in disaster-affected areas.
  arXiv  Detail & Related papers  (2024-06-12T09:21:28Z)
• Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization [56.95046107046027]
  We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for scene coordinate regression. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain.
  arXiv  Detail & Related papers  (2023-10-10T20:11:13Z)
• AB2CD: AI for Building Climate Damage Classification and Detection [0.0]
  We explore the implementation of deep learning techniques for precise building damage assessment in the context of natural hazards. We tackle the challenges of generalization to novel disasters and regions while accounting for the influence of low-quality and noisy labels. Our research findings showcase the potential and limitations of advanced AI solutions in enhancing the impact assessment of climate change-induced extreme weather events.
  arXiv  Detail & Related papers  (2023-09-03T03:37:04Z)
• One-class Damage Detector Using Deeper Fully-Convolutional Data Descriptions for Civil Application [0.0]
  One-class damage detection approach has an advantage in that normal images can be used to optimize model parameters. We propose a civil-purpose application for automating one-class damage detection reproducing a fully convolutional data description (FCDD) as a baseline model.
  arXiv  Detail & Related papers  (2023-03-03T06:27:15Z)
• To Make Yourself Invisible with Adversarial Semantic Contours [47.755808439588094]
  Adversarial Semantic Contour (ASC) is an estimate of a Bayesian formulation of sparse attack with a deceived prior of object contour. We show that ASC can corrupt the prediction of 9 modern detectors with different architectures. We conclude with cautions about contour being the common weakness of object detectors with various architecture.
  arXiv  Detail & Related papers  (2023-03-01T07:22:39Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)
• Assessing out-of-domain generalization for robust building damage detection [78.6363825307044]
  Building damage detection can be automated by applying computer vision techniques to satellite imagery. Models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. We argue that future work should focus on the OOD regime instead.
  arXiv  Detail & Related papers  (2020-11-20T10:30:43Z)
• Learning from Multimodal and Multitemporal Earth Observation Data for Building Damage Mapping [17.324397643429638]
  We have developed a global multisensor and multitemporal dataset for building damage mapping. The global dataset contains high-resolution optical imagery and high-to-moderate-resolution multiband SAR data. We defined a damage mapping framework for the semantic segmentation of damaged buildings based on a deep convolutional neural network algorithm.
  arXiv  Detail & Related papers  (2020-09-14T05:04:19Z)
• Comprehensive Semantic Segmentation on High Resolution UAV Imagery for Natural Disaster Damage Assessment [0.26249027950824505]
  We present a large-scale hurricane Michael dataset for visual perception in disaster scenarios. We analyze state-of-the-art deep neural network models for semantic segmentation.
  arXiv  Detail & Related papers  (2020-09-02T17:07:28Z)
• RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery [83.49145695899388]
  RescueNet is a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-to-end. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods.
  arXiv  Detail & Related papers  (2020-04-15T19:52:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.