RAPTOR-AI for Disaster OODA Loop: Hierarchical Multimodal RAG with Experience-Driven Agentic Decision-Making

• URL: http://arxiv.org/abs/2602.00030v1
• Date: Sun, 18 Jan 2026 10:58:41 GMT
• Title: RAPTOR-AI for Disaster OODA Loop: Hierarchical Multimodal RAG with Experience-Driven Agentic Decision-Making
• Abstract summary: This work introduces an agentic Retrieval-Augmented Generation framework designed to support the three canonical phases of disaster response: initial rescue, mid-term recovery, and long-term reconstruction.<n>To achieve robust multimodal grounding, we construct a hierarchical knowledge base that integrates textual disaster manuals, historical lessons, and aerial and ground-level imagery.<n>Our system builds on the open-source multimodal implementation, which processes 46 tsunami-related PDFs (2,378 pages) using BLIP-based image captioning, ColVBERT embeddings, and long-context summarization.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective humanitarian assistance and disaster relief (HADR) requires rapid situational understanding, reliable decision support, and the ability to generalize across diverse and previously unseen disaster contexts. This work introduces an agentic Retrieval-Augmented Generation (RAG) framework designed to support the three canonical phases of disaster response: initial rescue, mid-term recovery, and long-term reconstruction. To achieve robust multimodal grounding, we construct a hierarchical knowledge base that integrates textual disaster manuals, historical lessons (e.g., the 2011 Tohoku earthquake), and both aerial and ground-level imagery. Our system builds on the open-source multimodal implementation, which processes 46 tsunami-related PDFs (2,378 pages) using BLIP-based image captioning, ColVBERT embeddings, and long-context summarization to generate an efficient, structured multimodal retrieval tree optimized for disaster knowledge preservation. An agentic controller dynamically selects retrieval strategies (e.g., RAPTOR, ColBERT) through entropy-aware scene abstraction, enabling adaptive reasoning across heterogeneous inputs. Additionally, a lightweight LoRA-based post-training method injects experiential knowledge from past disasters, enhancing the models' capacity to support both expert and non-expert responders. Experiments on real disaster datasets demonstrate improved situational grounding, enhanced task decomposition accuracy, and superior usability for emergency operations. Incorporating recent advances in long-context RAG systems, agentic information retrieval, and contemporary emergency response AI, our system achieves substantial gains through adaptive retrieval-augmented generation with self-reasoning and multimodal chain-of-thought capabilities.

Related papers

• RAPID: A Reproducible Multi-Agent Pipeline for Interpretable Disaster Damage Assessment from Satellite and Street-View Imagery [22.976017696905092]
  RAPID is a reproducible multi-agent pipeline for interpretable disaster damage assessment.<n>It coordinates specialized agents to perform cross-view understanding, image restoration, structured damage recognition, and geographical reasoning.<n>It produces fine-grained, interpretable assessments and automatically generates location-specific, decision-relevant disaster reports.
  arXiv  Detail & Related papers  (2026-06-20T01:05:44Z)
• DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning [58.09293442211248]
  Deep Research (DR) has emerged as a new agentic paradigm to tackle complex, open-ended research tasks.<n>This report presents DuMate-DeepResearch, a multi-agent DR framework built on the Qianfan Agent Foundry.
  arXiv  Detail & Related papers  (2026-06-05T14:10:48Z)
• DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments [18.846222887851713]
  DisasterBench is a multi-stage multimodal reasoning benchmark for UAV-Based disaster response in complex environments.<n>DisasterVL is a lightweight multimodal model optimized with a three-stage pipeline combining domain instruction tuning, chain-of-thought-guided multimodal alignment, and reinforcement learning-based policy optimization.
  arXiv  Detail & Related papers  (2026-06-04T14:31:11Z)
• Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations [55.251494694783894]
  Disaster Operational Response Agent benchmark (DORA) is the first agentic benchmark for end-to-end disaster response.<n>Tasks span five dimensions that cover the operational disaster-response pipeline: disaster perception, spatial relational analysis, rescue and evacuation planning, temporal evolution reasoning, and multi-modal report synthesis.<n>DORA establishes a rigorous testbed for operationally reliable disaster-response agents.
  arXiv  Detail & Related papers  (2026-05-12T06:57:41Z)
• ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding [26.48338017389645]
  ChangeQuery is a unified multimodal framework designed for comprehensive, all-weather disaster situation awareness.<n>It is trained on structured data to provide precise damage quantification, region-specific descriptions, and holistic post-disaster summaries.<n>Experiments demonstrate that ChangeQuery establishes a new state-of-the-art, providing a robust and interpretable solution for complex disaster monitoring.
  arXiv  Detail & Related papers  (2026-04-24T08:05:39Z)
• EpiAgent: An Agent-Centric System for Ancient Inscription Restoration [60.77886293107454]
  EpiAgent is an agent-centric system that formulates inscription restoration as a hierarchical planning problem.<n>EpiAgent achieves superior restoration quality and stronger generalization compared to existing methods.<n>Our work marks an important step toward expert-level agent-driven restoration of cultural heritage.
  arXiv  Detail & Related papers  (2026-04-10T14:37:54Z)
• AgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Research [85.51475655916026]
  AgentCPM-Report is a lightweight yet high-performing local solution composed of a framework that mirrors the human writing process.<n>Our framework uses a Writing As Reasoning Policy (WARP), which enables models to dynamically revise outlines.<n>Experiments on DeepResearch Bench, DeepConsult, and DeepResearch Gym demonstrate that AgentCPM-Report outperforms leading closed-source systems.
  arXiv  Detail & Related papers  (2026-02-06T09:45:04Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• Multi-hop Reasoning via Early Knowledge Alignment [68.28168992785896]
  Early Knowledge Alignment (EKA) aims to align Large Language Models with contextually relevant retrieved knowledge.<n>EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency.<n>EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models.
  arXiv  Detail & Related papers  (2025-12-23T08:14:44Z)
• ReCAP: Recursive Context-Aware Reasoning and Planning for Large Language Model Agents [61.51091799997476]
  We introduce ReCAP (Recursive Context-Aware Reasoning and Planning), a hierarchical framework with shared context for reasoning and planning in large language models (LLMs)<n>ReCAP combines three key mechanisms: plan-ahead decomposition, structured re-injection of parent plans, and memory-efficient execution.<n>Experiments demonstrate that ReCAP substantially improves subgoal alignment and success rates on various long-horizon reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-27T20:03:55Z)
• Disaster Management in the Era of Agentic AI Systems: A Vision for Collective Human-Machine Intelligence for Augmented Resilience [13.155469498169866]
  Disaster Copilot is a vision for a multi-agent artificial intelligence system designed to overcome these systemic challenges.<n>The proposed architecture utilizes a central orchestrator to coordinate diverse sub-agents, each specializing in critical domains such as predictive risk analytics, situational awareness, and impact assessment.
  arXiv  Detail & Related papers  (2025-10-16T02:02:10Z)
• WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research [73.58638285105971]
  This paper tackles textbfopen-ended deep research (OEDR), a complex challenge where AI agents must synthesize vast web-scale information into insightful reports.<n>We introduce textbfWebWeaver, a novel dual-agent framework that emulates the human research process.<n>Our framework establishes a new state-of-the-art across major OEDR benchmarks, including DeepResearch Bench, DeepConsult, and DeepResearchGym.
  arXiv  Detail & Related papers  (2025-09-16T17:57:21Z)
• A Multimodal RAG Framework for Housing Damage Assessment: Collaborative Optimization of Image Encoding and Policy Vector Retrieval [0.0]
  After natural disasters, accurate evaluations of damage to housing are important for insurance claims response and planning of resources.<n>In this work, we introduce a novel multimodal retrieval-augmented generation (MM-RAG) framework.<n>The framework takes end-to-end training, and combines the comparison loss, the retrieval loss and the generation loss to form multi-task optimization objectives.
  arXiv  Detail & Related papers  (2025-09-10T01:58:07Z)
• HiRA: A Hierarchical Reasoning Framework for Decoupled Planning and Execution in Deep Search [85.12447821237045]
  HiRA is a hierarchical framework that separates strategic planning from specialized execution.<n>Our approach decomposes complex search tasks into focused subtasks, assigns each subtask to domain-specific agents equipped with external tools and reasoning capabilities.<n> Experiments on four complex, cross-modal deep search benchmarks demonstrate that HiRA significantly outperforms state-of-the-art RAG and agent-based systems.
  arXiv  Detail & Related papers  (2025-07-03T14:18:08Z)
• A Deep Learning framework for building damage assessment using VHR SAR and geospatial data: demonstration on the 2023 Turkiye Earthquake [1.6070833439280312]
  Building damage identification shortly after a disaster is crucial for guiding emergency response and recovery efforts.<n>We introduce a novel multimodal deep learning (DL) framework for detecting building damage using single-date very high resolution (VHR) Synthetic Aperture Radar (SAR) imagery.<n>Our method integrates SAR image patches, OpenStreetMap (OSM) building footprints, digital surface model (DSM) data, and structural and exposure attributes from the Global Earthquake Model (GEM)<n>Results highlight that incorporating geospatial features significantly enhances detection performance and generalizability to previously unseen areas.
  arXiv  Detail & Related papers  (2025-06-27T15:49:58Z)
• TRIFFID: Autonomous Robotic Aid For Increasing First Responders Efficiency [33.272295358597695]
  This paper introduces the TRIFFID system, which integrates unmanned ground and aerial vehicles with advanced artificial intelligence functionalities.<n>The proposed system enhances emergency response teams by providing advanced mission planning, safety monitoring, and adaptive task execution capabilities.
  arXiv  Detail & Related papers  (2025-02-13T14:46:40Z)
• CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics [50.122541222825156]
  This study introduces a novel approach to disaster text classification by enhancing a pre-trained Large Language Model (LLM)<n>Our methodology involves creating a comprehensive instruction dataset from disaster-related tweets, which is then used to fine-tune an open-source LLM.<n>This fine-tuned model can classify multiple aspects of disaster-related information simultaneously, such as the type of event, informativeness, and involvement of human aid.
  arXiv  Detail & Related papers  (2024-06-16T23:01:10Z)
• Improving Community Resiliency and Emergency Response With Artificial Intelligence [0.05541644538483946]
  We are working towards a multipronged emergency response tool that provide stakeholders timely access to comprehensive, relevant, and reliable information. Our tool consists of encoding multiple layers of open source geospatial data including flood risk location, road network strength, inundation maps that proxy inland flooding and computer vision semantic segmentation for estimating flooded areas and damaged infrastructure. These data layers are combined and used as input data for machine learning algorithms such as finding the best evacuation routes before, during and after an emergency or providing a list of available lodging for first responders in an impacted area for first.
  arXiv  Detail & Related papers  (2020-05-28T18:05:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.