DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments

• URL: http://arxiv.org/abs/2606.06217v1
• Date: Thu, 04 Jun 2026 14:31:11 GMT
• Title: DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments
• Abstract summary: DisasterBench is a multi-stage multimodal reasoning benchmark for UAV-Based disaster response in complex environments.<n>DisasterVL is a lightweight multimodal model optimized with a three-stage pipeline combining domain instruction tuning, chain-of-thought-guided multimodal alignment, and reinforcement learning-based policy optimization.
• Score: 18.846222887851713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When a disaster unfolds, responders must answer not only what is happening, but also why it is happening, what will happen next, and what to do now, often from noisy low-altitude UAV views and under tight on-site compute constraints. However, most existing multimodal benchmarks emphasize perception (e.g., recognition/description), cover limited disaster types, and provide insufficient support for the multi-stage reasoning required in practical emergency response. We introduce DisasterBench, a multi-stage multimodal reasoning benchmark for UAV-Based disaster response in complex environments. DisasterBench spans 14 disaster-related scene types and 9 response-critical tasks across pre-, during-, and post-disaster stages, with fine-grained disaster-task mappings that explicitly test causal attribution, propagation prediction, damage analysis, and decision-oriented reasoning. To enable reasoning on the edge, we further propose DisasterVL, a lightweight multimodal model optimized with a three-stage pipeline combining domain instruction tuning, chain-of-thought-guided multimodal alignment, and reinforcement learning-based policy optimization. Experiments across 21 popular MLLMs show that our 2B-parameter DisasterVL outperforms all evaluated open-source models and substantially narrows the gap to state-of-the-art closed-source models, achieving GPT-4o-comparable reasoning accuracy with superior efficiency. The project page is available at https://github.com/TanmouTT/DisasterBench.

Related papers

• RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistants [48.63201300674347]
  Multimodal industrial anomaly inspection assistants are a critical component of next-generation smart factories.<n>MSLMs offer a deployable alternative, yet progress is constrained by the lack of comprehensive robustness analyses.<n>We develop RobustMAD to evaluate model robustness through diverse open-ended queries.
  arXiv  Detail & Related papers  (2026-06-26T09:15:07Z)
• DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints [29.573427850164222]
  Disasters cause severe societal impacts, demanding rapid coordination of heterogeneous AI tools.<n>Effective coordination requires more than selecting semantically plausible tools.<n>We introduce DisasterBench, a benchmark for evaluating structured multi-agent planning.
  arXiv  Detail & Related papers  (2026-05-27T04:50:23Z)
• Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations [55.251494694783894]
  Disaster Operational Response Agent benchmark (DORA) is the first agentic benchmark for end-to-end disaster response.<n>Tasks span five dimensions that cover the operational disaster-response pipeline: disaster perception, spatial relational analysis, rescue and evacuation planning, temporal evolution reasoning, and multi-modal report synthesis.<n>DORA establishes a rigorous testbed for operationally reliable disaster-response agents.
  arXiv  Detail & Related papers  (2026-05-12T06:57:41Z)
• CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models [48.33850939857255]
  We develop a theoretical framework that links sampling coverage, instance difficulty, and residual risk.<n>Our analysis reveals that multimodal reasoning exhibits a heavy-tailed difficulty distribution.<n>We propose Coverage-Aware Multimodal Decoding (CAMD), an adaptive inference mechanism that dynamically allocates according to estimated uncertainty.
  arXiv  Detail & Related papers  (2026-03-16T02:31:03Z)
• TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models [52.21298691935726]
  The ability to reason over time series is a fundamental skill for generalist models to solve practical problems.<n>To bridge this gap, we introduce TSRBench, a comprehensive benchmark designed to stress-test the full spectrum of time series reasoning capabilities.
  arXiv  Detail & Related papers  (2026-01-26T18:04:54Z)
• DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment [19.434058305975167]
  DisasterInsight is a benchmark designed to evaluate vision-language models (VLMs) on realistic disaster analysis tasks.<n>It restructures the xBD dataset into approximately 112K building-centered instances.<n>It supports instruction-diverse evaluation across multiple tasks, including building-function classification, damage-level and disaster-type classification, counting, and structured report generation aligned with humanitarian assessment guidelines.
  arXiv  Detail & Related papers  (2026-01-26T13:48:11Z)
• RAPTOR-AI for Disaster OODA Loop: Hierarchical Multimodal RAG with Experience-Driven Agentic Decision-Making [0.0]
  This work introduces an agentic Retrieval-Augmented Generation framework designed to support the three canonical phases of disaster response: initial rescue, mid-term recovery, and long-term reconstruction.<n>To achieve robust multimodal grounding, we construct a hierarchical knowledge base that integrates textual disaster manuals, historical lessons, and aerial and ground-level imagery.<n>Our system builds on the open-source multimodal implementation, which processes 46 tsunami-related PDFs (2,378 pages) using BLIP-based image captioning, ColVBERT embeddings, and long-context summarization.
  arXiv  Detail & Related papers  (2026-01-18T10:58:41Z)
• Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method [96.63801368613177]
  We present a new task that elevates video anomaly analysis from descriptive understanding to structured, multi-stage reasoning.<n>We present a new dataset with 8,641 videos, totaling more than 50,000 samples, making it one of the largest datasets for video anomaly understanding.<n>Building upon the proposed task and dataset, we develop an end-to-end MLLM-based VAR model termed Vad-R1-Plus, which supports adaptive hierarchical reasoning and risk-aware decision making.
  arXiv  Detail & Related papers  (2026-01-15T08:09:04Z)
• Think First, Assign Next (ThiFAN-VQA): A Two-stage Chain-of-Thought Framework for Post-Disaster Damage Assessment [0.0]
  We propose ThiFAN-VQA, a two-stage reasoning-based framework for visual question answering (VQA) in disaster scenarios.<n>By integrating a custom information retrieval system, domain-specific prompting, and reasoning-guided answer selection, ThiFAN-VQA bridges the gap between zero-shot and supervised methods.<n> Experiments on FloodNet and RescueNet-VQA, UAV-based datasets from flood- and hurricane-affected regions, demonstrate that ThiFAN-VQA achieves superior accuracy, interpretability, and adaptability.
  arXiv  Detail & Related papers  (2025-11-24T14:32:07Z)
• Directional Reasoning Injection for Fine-Tuning MLLMs [51.53222423215055]
  Multimodal large language models (MLLMs) are rapidly advancing, yet their reasoning ability often lags behind that of strong text-only counterparts.<n>Existing methods to bridge this gap rely on supervised fine-tuning over large-scale multimodal reasoning data or reinforcement learning.<n>We propose Directional Reasoning Injection for Fine-Tuning (DRIFT) to solve this problem.
  arXiv  Detail & Related papers  (2025-10-16T18:06:46Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• A Multi-constraint and Multi-objective Allocation Model for Emergency Rescue in IoT Environment [3.8572535126902676]
  We've developed the Multi-Objective Shuffled Gray Froging Model (MSGWFLM) This multi-objective resource allocation model has been rigorously tested against 28 diverse challenges. It's effectiveness is particularly notable in complex, multi-cycle emergency rescue scenarios.
  arXiv  Detail & Related papers  (2024-03-15T13:42:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.