MAD: Mapping-Aware World Models for Agile Quadrotor Flight

• URL: http://arxiv.org/abs/2606.04534v1
• Date: Wed, 03 Jun 2026 07:17:57 GMT
• Title: MAD: Mapping-Aware World Models for Agile Quadrotor Flight
• Abstract summary: Mapping-Aware Dreamer (MAD) is a geometry-aware world model for vision-based quadrotor flight.<n>MAD learns recurrent latent dynamics that reconstruct robocentric occupancy and visibility grid maps.<n>We demonstrate safe indoor and outdoor flight under limited sensing, reaching 9.66 m/s in simulation and 5.05 m/s in real-world forest experiments.
• Score: 47.458890396048126
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agile quadrotor flight in cluttered scenes requires more than a reactive mapping from a depth image to a control command: the vehicle must remember which regions have been observed, infer nearby occupied space, and act under partial visibility and tight latency. In this paper, we present Mapping-Aware Dreamer (MAD), a geometry-aware world model for vision-based quadrotor flight. Instead of using raw-image reconstruction as the main self-supervised objective, MAD learns recurrent latent dynamics that reconstruct robocentric occupancy and visibility grid maps together with proprioceptive states. This design forces the latent state to encode local geometry, visibility history, and ego-motion in a form that is directly relevant to collision avoidance. MAD is trained in DiffAero using a GPU-parallel map-construction module that provides high-throughput supervision for occupancy and visibility. The learned representation is used in three policy-learning modes: imagination-based MAD-Dreamer and feature-extractor variants based on PPO and SHAC. Across visual navigation and racing tasks, MAD-based agents achieve higher success rates, faster flight, and better cross-task transfer than corresponding vision-only baselines. The model also produces interpretable map predictions and accurate ego-motion estimates from depth observations. We further deploy the learned policy on a physical quadrotor with an Intel RealSense D435i and demonstrate safe indoor and outdoor flight under limited sensing, reaching 9.66 m/s in simulation and 5.05 m/s in real-world forest experiments. These results show that mapping-aware world models provide a practical middle ground between modular aerial navigation and end-to-end learning.

Related papers

• AirForesight: Current-to-Future Spatial Map Imagination with Cross-Space Planning Consistency for UAV-VLN [28.176475934194638]
  Unmanned Aerial Vehicle Vision-Language Navigation (UAV-VLN) requires agents to follow language instructions.<n>AirForesight is a current-to-future spatial map imagination framework for UAV-VLN.
  arXiv  Detail & Related papers  (2026-08-13T05:18:23Z)
• AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight [42.75555298269899]
  Language-conditioned quadrotor flight requires a policy to ground semantic goals.<n>We present AeroAct, an action-centered world-action model (WAM) for quadrotor navigation.
  arXiv  Detail & Related papers  (2026-07-16T13:46:00Z)
• MotionScape: A Large-Scale Real-World Highly Dynamic UAV Video Dataset for World Models [6.680381992984212]
  We present MotionScape, a large-scale real-world UAVview video dataset with highly dynamic motion for world modeling.<n>MotionScape contains over 30 hours of 4K UAV-view videos, totaling more than 4.5M frames.<n>This novel dataset features semantically and geometrically aligned training samples, where diverse real-world UAV videos are tightly coupled with accurate 6-DoF camera trajectories and fine-grained natural language descriptions.
  arXiv  Detail & Related papers  (2026-04-09T08:59:40Z)
• Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Space [48.19308247102762]
  We propose ANWM, an aerial navigation world model that predicts future visual observations conditioned on past frames and actions.<n> ANWM is trained on 4-DoF UAV trajectories and introduces a physics-inspired module: Future Frame Projection.<n> Empirical results demonstrate that ANWM significantly outperforms existing world models in long-distance visual forecasting and improves UAV navigation success rates in large-scale environments.
  arXiv  Detail & Related papers  (2025-12-26T06:22:39Z)
• Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation [54.04601077224252]
  Embodied scene understanding requires not only comprehending visual-spatial information but also determining where to explore next in the 3D physical world.<n>underlinetextbf3D vision-language learning enables embodied agents to effectively explore and understand their environment.<n>model's versatility enables navigation using diverse input modalities, including categories, language descriptions, and reference images.
  arXiv  Detail & Related papers  (2025-07-05T14:15:52Z)
• Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigation [11.267956604072845]
  Aerial Vision-and-Language Navigation (VLN) is a novel task enabling Unmanned Aerial Vehicles (UAVs) to navigate in outdoor environments through natural language instructions and visual cues.<n>We propose a training-free, zero-shot framework for aerial VLN tasks, where the large language model (LLM) is leveraged as the agent for action prediction.
  arXiv  Detail & Related papers  (2024-10-11T03:54:48Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• Self-Supervised Object Goal Navigation with In-Situ Finetuning [110.6053241629366]
  This work builds an agent that builds self-supervised models of the world via exploration. We identify a strong source of self-supervision that can train all components of an ObjectNav agent. We show that our agent can perform competitively in the real world and simulation.
  arXiv  Detail & Related papers  (2022-12-09T03:41:40Z)
• Occupancy Anticipation for Efficient Exploration and Navigation [97.17517060585875]
  We propose occupancy anticipation, where the agent uses its egocentric RGB-D observations to infer the occupancy state beyond the visible regions. By exploiting context in both the egocentric views and top-down maps our model successfully anticipates a broader map of the environment. Our approach is the winning entry in the 2020 Habitat PointNav Challenge.
  arXiv  Detail & Related papers  (2020-08-21T03:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.