From Pixels to Semantics: A Multi-Stage AI Framework for Structural Damage Detection in Satellite Imagery

• URL: http://arxiv.org/abs/2603.22768v1
• Date: Tue, 24 Mar 2026 03:50:41 GMT
• Title: From Pixels to Semantics: A Multi-Stage AI Framework for Structural Damage Detection in Satellite Imagery
• Abstract summary: We propose a novel framework that integrates AI-based super-resolution, deep learning object detection, and Vision-Language Models (VLMs) for comprehensive post-disaster building damage assessment.<n>First, we enhance pre- and post-disaster satellite imagery using a Video Restoration Transformer (VRT) to upscale images from 1024x1024 to 4096x4096 resolution.<n>We employ CLIPScore for reference-free semantic alignment and introduce a multi-model VLM-as-a-Jury strategy to reduce individual model bias in safety-critical decision making.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rapid and accurate structural damage assessment following natural disasters is critical for effective emergency response and recovery. However, remote sensing imagery often suffers from low spatial resolution, contextual ambiguity, and limited semantic interpretability, reducing the reliability of traditional detection pipelines. In this work, we propose a novel hybrid framework that integrates AI-based super-resolution, deep learning object detection, and Vision-Language Models (VLMs) for comprehensive post-disaster building damage assessment. First, we enhance pre- and post-disaster satellite imagery using a Video Restoration Transformer (VRT) to upscale images from 1024x1024 to 4096x4096 resolution, improving structural detail visibility. Next, a YOLOv11-based detector localizes buildings in pre-disaster imagery, and cropped building regions are analyzed using VLMs to semantically assess structural damage across four severity levels. To ensure robust evaluation in the absence of ground-truth captions, we employ CLIPScore for reference-free semantic alignment and introduce a multi-model VLM-as-a-Jury strategy to reduce individual model bias in safety-critical decision making. Experiments on subsets of the xBD dataset, including the Moore Tornado and Hurricane Matthew events, demonstrate that the proposed framework enhances the semantic interpretation of damaged buildings. In addition, our framework provides helpful recommendations to first responders for recovery based on damage analysis.

Related papers

• DamageScope: Vision-Language Retrieval at Scale for Disaster Damage Assessment from Satellite Imagery [6.514592086157328]
  DamageScope is a retrieval-augmented framework that combines satellite imagery with Vision-Language Models (VLMs) and Large Language Models (LLMs)<n>Built on a Retrieval-Augmented Generation (RAG) framework, DamageScope extracts structured visual representations from satellite imagery to support interactive natural language queries for damage assessment.
  arXiv  Detail & Related papers  (2026-08-21T18:09:21Z)
• RAPID: A Reproducible Multi-Agent Pipeline for Interpretable Disaster Damage Assessment from Satellite and Street-View Imagery [22.976017696905092]
  RAPID is a reproducible multi-agent pipeline for interpretable disaster damage assessment.<n>It coordinates specialized agents to perform cross-view understanding, image restoration, structured damage recognition, and geographical reasoning.<n>It produces fine-grained, interpretable assessments and automatically generates location-specific, decision-relevant disaster reports.
  arXiv  Detail & Related papers  (2026-06-20T01:05:44Z)
• Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback [51.08692072066352]
  Text-to-image (T2I) models still exhibit localized, subtle, and structurally complex failures.<n>Structured Defect Grounding casts T2I diagnosis as structured set prediction by modeling each defect as a location, type, reason, importance.<n>Our detector outperforms leading proprietary VLMs on structured defect grounding.
  arXiv  Detail & Related papers  (2026-06-04T13:03:16Z)
• Satellite to Street : Disaster Impact Estimator [0.12938914787881173]
  The present work proposes a deep-learning framework that jointly processes pre- and post-disaster satellite images to obtain fine-grained pixel-level damage maps: Satellite-to-Street: Disaster Impact Estimator.<n>The model uses a modified dual-input U-Net architecture with enhanced feature fusion to capture both the local structural changes as well as the broader contextual cues.
  arXiv  Detail & Related papers  (2025-11-24T06:20:40Z)
• Structural Damage Detection Using AI Super Resolution and Visual Language Model [0.0]
  This study proposes a novel, cost-effective framework that leverages aerial drone footage, an advanced AI-based video super-resolution model, Video Restoration Transformer (VRT), and Gemma3:27b, a 27 billion parameter Visual Language Model (VLM)<n>This integrated system is designed to improve low-resolution disaster footage, identify structural damage, and classify buildings into four damage categories, ranging from no/slight damage to total destruction, along with associated risk levels.<n>The framework achieved a classification accuracy of 84.5%, demonstrating its ability to provide highly accurate results.
  arXiv  Detail & Related papers  (2025-08-23T20:12:06Z)
• Multiclass Post-Earthquake Building Assessment Integrating High-Resolution Optical and SAR Satellite Imagery, Ground Motion, and Soil Data with Transformers [0.0]
  We introduce a framework that combines high-resolution post-earthquake satellite imagery with building-specific metadata relevant to the seismic performance of the structure.<n>Our model achieves state-of-the-art performance in multiclass post-earthquake damage identification for buildings from the Turkey-Syria earthquake on February 6, 2023.
  arXiv  Detail & Related papers  (2024-12-05T23:19:51Z)
• Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Data [66.49494950674402]
  We leverage emerging text-to-image generative models in creating large-scale synthetic supervision for the task of damage assessment from aerial images. We build an efficient and easily scalable pipeline to generate thousands of post-disaster images from low-resource domains. We validate the strength of our proposed framework under cross-geography domain transfer setting from xBD and SKAI images in both single-source and multi-source settings.
  arXiv  Detail & Related papers  (2024-05-22T16:07:05Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• Exploring Resolution and Degradation Clues as Self-supervised Signal for Low Quality Object Detection [77.3530907443279]
  We propose a novel self-supervised framework to detect objects in degraded low resolution images. Our methods has achieved superior performance compared with existing methods when facing variant degradation situations.
  arXiv  Detail & Related papers  (2022-08-05T09:36:13Z)
• RestoreDet: Degradation Equivariant Representation for Object Detection in Low Resolution Images [81.91416537019835]
  We propose a novel framework, RestoreDet, to detect objects in degraded low resolution images. Our framework based on CenterNet has achieved superior performance compared with existing methods when facing variant degradation situations.
  arXiv  Detail & Related papers  (2022-01-07T03:40:23Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery [83.49145695899388]
  RescueNet is a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-to-end. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods.
  arXiv  Detail & Related papers  (2020-04-15T19:52:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.