FIRE-VLM: A Vision-Language-Driven Reinforcement Learning Framework for UAV Wildfire Tracking in a Physics-Grounded Fire Digital Twin

• URL: http://arxiv.org/abs/2601.03449v1
• Date: Tue, 06 Jan 2026 22:31:57 GMT
• Title: FIRE-VLM: A Vision-Language-Driven Reinforcement Learning Framework for UAV Wildfire Tracking in a Physics-Grounded Fire Digital Twin
• Abstract summary: Wildfire monitoring demands autonomous systems capable of reasoning under extreme visual degradation.<n>Fire-VLM is the first end-to-end vision-language model (VLM) guided reinforcement learning framework trained entirely within a wildfire digital twin.<n>Our contributions are: (1) a GIS-to-simulation pipeline for constructing wildfire digital twins; (2) a VLM-guided RL agent for UAV firefront tracking; and (3) a wildfire-aware reward design that combines physical terms with VLM semantics.
• Score: 5.958733922398943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Wildfire monitoring demands autonomous systems capable of reasoning under extreme visual degradation, rapidly evolving physical dynamics, and scarce real-world training data. Existing UAV navigation approaches rely on simplified simulators and supervised perception pipelines, and lack embodied agents interacting with physically realistic fire environments. We introduce FIRE-VLM, the first end-to-end vision-language model (VLM) guided reinforcement learning (RL) framework trained entirely within a high-fidelity, physics-grounded wildfire digital twin. Built from USGS Digital Elevation Model (DEM) terrain, LANDFIRE fuel inventories, and semi-physical fire-spread solvers, this twin captures terrain-induced runs, wind-driven acceleration, smoke plume occlusion, and dynamic fuel consumption. Within this environment, a PPO agent with dual-view UAV sensing is guided by a CLIP-style VLM. Wildfire-specific semantic alignment scores, derived from a single prompt describing active fire and smoke plumes, are integrated as potential-based reward shaping signals. Our contributions are: (1) a GIS-to-simulation pipeline for constructing wildfire digital twins; (2) a VLM-guided RL agent for UAV firefront tracking; and (3) a wildfire-aware reward design that combines physical terms with VLM semantics. Across five digital-twin evaluation tasks, our VLM-guided policy reduces time-to-detection by up to 6 times, increases time-in-FOV, and is, to our knowledge, the first RL-based UAV wildfire monitoring system demonstrated in kilometer-scale, physics-grounded digital-twin fires.

Related papers

• Multi-Agent Reinforcement Learning for Autonomous UAV Exploration in Wildfire Response [0.30586855806896035]
  This study develops a deep reinforcement learning framework for training Unmanned Aerial Vehicle (UAV) agents to navigate and monitor simulated wildfire environments.<n>Results show that agents learn increasingly stable and effective behaviors over time, as demonstrated by converging loss trends, improved reward signals, and more consistent navigation patterns such as fire-boundary tracking.
  arXiv  Detail & Related papers  (2026-09-09T16:45:19Z)
• DynFly: Dynamic-Aware Continuous Trajectory Generation for UAV Vision-Language Navigation in Urban Environments [55.73595346639892]
  We propose DynFly to bridge high-level navigation reasoning and executable UAV motion.<n>DynFly represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation.<n>Experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality.
  arXiv  Detail & Related papers  (2026-06-30T13:33:48Z)
• DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA [72.9197085473598]
  DIAL is a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck.<n>A VLM-based System-2 performs latent world modeling by latent visual foresight within the VLM's native feature space.<n>A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions.
  arXiv  Detail & Related papers  (2026-03-31T15:02:27Z)
• FireRescue: A UAV-Based Dataset and Enhanced YOLO Model for Object Detection in Fire Rescue Scenes [24.465732878368545]
  This paper first constructs a new dataset named "FireRescue" for rescue command, which covers multiple rescue scenarios.<n> Secondly, to tackle the problems of inter-class confusion and missed detection of small targets, this paper proposes an improved model named FRS-YOLO.
  arXiv  Detail & Related papers  (2025-12-31T04:37:51Z)
• UAV-MM3D: A Large-Scale Synthetic Benchmark for 3D Perception of Unmanned Aerial Vehicles with Multi-Modal Data [47.317955428393134]
  We introduce UAV-MM3D, a high-fidelity multimodal synthetic dataset for low-altitude UAV perception and motion understanding.<n>It comprises 400K synchronized frames across diverse scenes (urban areas, suburbs, forests, coastal regions) and weather conditions.<n>Each frame provides 2D/3D bounding boxes, 6-DoF poses, and instance-level annotations, enabling core tasks related to UAVs such as 3D detection, pose estimation, target tracking, and short-term trajectory forecasting.
  arXiv  Detail & Related papers  (2025-11-27T12:30:28Z)
• Trajectory Design for UAV-Based Low-Altitude Wireless Networks in Unknown Environments: A Digital Twin-Assisted TD3 Approach [62.11847362756054]
  Unmanned aerial vehicles (UAVs) are emerging as key enablers for low-altitude wireless network (LAWN)<n>We propose a digital twin (DT)-assisted training and deployment framework.<n>In this framework, the UAV transmits integrated sensing and communication signals to provide communication services to ground users, while simultaneously collecting echoes that are uploaded to the DT server to progressively construct virtual environments (VEs)<n>These VEs accelerate model training and are continuously updated with real-time UAV sensing data during deployment, supporting decision-making and enhancing flight safety.
  arXiv  Detail & Related papers  (2025-10-28T10:05:53Z)
• LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks [57.27815890269697]
  This work focuses on maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under energy constraints.<n>We introduce a Large Language Model (LLM)-guided multi-agent learning approach.<n>Results show that our method outperforms existing baselines in secrecy and energy efficiency.
  arXiv  Detail & Related papers  (2025-07-23T04:22:57Z)
• Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
  We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
  arXiv  Detail & Related papers  (2025-03-06T07:36:06Z)
• BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents [56.33989853438012]
  We propose BEVWorld, a framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View latent space for holistic environment modeling.<n>The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• Hardware Acceleration for Real-Time Wildfire Detection Onboard Drone Networks [6.313148708539912]
  wildfire detection in remote and forest areas is crucial for minimizing devastation and preserving ecosystems. Drones offer agile access to remote, challenging terrains, equipped with advanced imaging technology. limited computation and battery resources pose challenges in implementing and efficient image classification models. This paper aims to develop a real-time image classification and fire segmentation model.
  arXiv  Detail & Related papers  (2024-01-16T04:16:46Z)
• A comprehensive survey of research towards AI-enabled unmanned aerial systems in pre-, active-, and post-wildfire management [6.043705525669726]
  Wildfires have emerged as one of the most destructive natural disasters worldwide, causing catastrophic losses in both human lives and forest wildlife. Recently, the use of Artificial Intelligence (AI) in wildfires, propelled by the integration of Unmanned Aerial Vehicles (UAVs) and deep learning models, has created an unprecedented momentum to implement and develop more effective wildfire management.
  arXiv  Detail & Related papers  (2024-01-04T05:09:35Z)
• Optimal Wildfire Escape Route Planning for Drones under Dynamic Fire and Smoke [3.9561033879611944]
  The utilization of unmanned aerial vehicles (UAVs) has shown promise in aiding wildfire management efforts. This work focuses on the development of an optimal wildfire escape route planning system specifically designed for drones.
  arXiv  Detail & Related papers  (2023-12-06T14:30:15Z)
• Active Wildfires Detection and Dynamic Escape Routes Planning for Humans through Information Fusion between Drones and Satellites [3.9561033879611944]
  This paper proposes a fusion of UAV vision technology and satellite image analysis technology for active wildfires detection and road networks extraction. Fire source location and the segmentation of smoke and flames are targeted based on Sentinel 2 satellite imagery. The results demonstrate that the dynamic escape route planning algorithm can provide an optimal real-time navigation path for humans in the presence of fire.
  arXiv  Detail & Related papers  (2023-12-06T14:25:47Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.