Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN

• URL: http://arxiv.org/abs/2408.06761v1
• Date: Tue, 13 Aug 2024 09:37:26 GMT
• Title: Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN
• Authors: Hao Li, Fabian Deuser, Wenping Yina, Xuanshu Luo, Paul Walther, Gengchen Mai, Wei Huang, Martin Werner,
• Abstract summary: We propose a novel disaster mapping framework, namely CVDisaster, to simultaneously address geolocalization and damage perception estimation. CVDisaster consists of two cross-view models, where CVDisaster-Geoloc refers to a cross-view geolocalization model. We show that CVDisaster can achieve highly competitive performance (over 80% for geolocalization and 75% for damage perception estimation) with even limited fine-tuning efforts.
• Score: 9.128051274958356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nature disasters play a key role in shaping human-urban infrastructure interactions. Effective and efficient response to natural disasters is essential for building resilience and a sustainable urban environment. Two types of information are usually the most necessary and difficult to gather in disaster response. The first information is about disaster damage perception, which shows how badly people think that urban infrastructure has been damaged. The second information is geolocation awareness, which means how people whereabouts are made available. In this paper, we proposed a novel disaster mapping framework, namely CVDisaster, aiming at simultaneously addressing geolocalization and damage perception estimation using cross-view Street-View Imagery (SVI) and Very High-Resolution satellite imagery. CVDisaster consists of two cross-view models, where CVDisaster-Geoloc refers to a cross-view geolocalization model based on a contrastive learning objective with a Siamese ConvNeXt image encoder, and CVDisaster-Est is a cross-view classification model based on a Couple Global Context Vision Transformer (CGCViT). Taking Hurricane IAN as a case study, we evaluate the CVDisaster framework by creating a novel cross-view dataset (CVIAN) and conducting extensive experiments. As a result, we show that CVDisaster can achieve highly competitive performance (over 80% for geolocalization and 75% for damage perception estimation) with even limited fine-tuning efforts, which largely motivates future cross-view models and applications within a broader GeoAI research community. The data and code are publicly available at: https://github.com/tum-bgd/CVDisaster.

Related papers

• Building Damage Assessment in Conflict Zones: A Deep Learning Approach Using Geospatial Sub-Meter Resolution Data [9.146968506196446]
  We build an annotated dataset with pre- and post-conflict images of the Ukrainian city of Mariupol. We then explore the transferability of the CNN models in both zero-shot and learning scenarios. This is the first study to use sub-meter resolution imagery to assess building damage in combat zones.
  arXiv  Detail & Related papers  (2024-10-07T07:26:38Z)
• Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model [17.016411785224317]
  We present DAVI (Disaster Assessment with VIsion foundation model), which overcomes domain disparities and detects structural damage without requiring ground-truth labels of the target region. DAVI integrates task-specific knowledge from a model trained on source regions with an image segmentation foundation model to generate pseudo labels of possible damage in the target region. It then employs a two-stage refinement process, targeting both the pixel and overall image, to more accurately pinpoint changes in disaster-struck areas.
  arXiv  Detail & Related papers  (2024-06-12T09:21:28Z)
• Classification of structural building damage grades from multi-temporal photogrammetric point clouds using a machine learning model trained on virtual laser scanning data [58.720142291102135]
  We present a novel approach to automatically assess multi-class building damage from real-world point clouds. We use a machine learning model trained on virtual laser scanning (VLS) data. The model yields high multi-target classification accuracies (overall accuracy: 92.0% - 95.1%)
  arXiv  Detail & Related papers  (2023-02-24T12:04:46Z)
• A General Purpose Neural Architecture for Geospatial Systems [142.43454584836812]
  We present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias. We envision how such a model may facilitate cooperation between members of the community.
  arXiv  Detail & Related papers  (2022-11-04T09:58:57Z)
• Assessing out-of-domain generalization for robust building damage detection [78.6363825307044]
  Building damage detection can be automated by applying computer vision techniques to satellite imagery. Models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. We argue that future work should focus on the OOD regime instead.
  arXiv  Detail & Related papers  (2020-11-20T10:30:43Z)
• Physics-informed GANs for Coastal Flood Visualization [65.54626149826066]
  We create a deep learning pipeline that generates visual satellite images of current and future coastal flooding. By evaluating the imagery relative to physics-based flood maps, we find that our proposed framework outperforms baseline models in both physical-consistency and photorealism. While this work focused on the visualization of coastal floods, we envision the creation of a global visualization of how climate change will shape our earth.
  arXiv  Detail & Related papers  (2020-10-16T02:15:34Z)
• Learning from Multimodal and Multitemporal Earth Observation Data for Building Damage Mapping [17.324397643429638]
  We have developed a global multisensor and multitemporal dataset for building damage mapping. The global dataset contains high-resolution optical imagery and high-to-moderate-resolution multiband SAR data. We defined a damage mapping framework for the semantic segmentation of damaged buildings based on a deep convolutional neural network algorithm.
  arXiv  Detail & Related papers  (2020-09-14T05:04:19Z)
• Improving Emergency Response during Hurricane Season using Computer Vision [0.06882042556551608]
  We have developed a framework for crisis response and management that incorporates the latest technologies in computer vision (CV), inland flood prediction, damage assessment and data visualization. Our computer-vision model analyzes spaceborne and airborne imagery to detect relevant features during and after a natural disaster. We have designed an ensemble of models to identify features including water, roads, buildings, and vegetation from the imagery.
  arXiv  Detail & Related papers  (2020-08-17T15:42:02Z)
• MSNet: A Multilevel Instance Segmentation Network for Natural Disaster Damage Assessment in Aerial Videos [74.22132693931145]
  We study the problem of efficiently assessing building damage after natural disasters like hurricanes, floods or fires. The first contribution is a new dataset, consisting of user-generated aerial videos from social media with annotations of instance-level building damage masks. The second contribution is a new model, namely MSNet, which contains novel region proposal network designs.
  arXiv  Detail & Related papers  (2020-06-30T02:23:05Z)
• RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery [83.49145695899388]
  RescueNet is a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-to-end. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods.
  arXiv  Detail & Related papers  (2020-04-15T19:52:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.