Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls

• URL: http://arxiv.org/abs/2605.19728v1
• Date: Tue, 19 May 2026 12:02:17 GMT
• Title: Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls
• Abstract summary: We propose textbfAero-World, a method for converting a pretrained image-to-video diffusion model into a controllable aerial video generator.<n>A frozen latent-space Physics Probe, trained independently on real video--IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning.<n>On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape.
• Score: 57.166020875486474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation video models produce visually impressive results, but their use in embodied AI remains limited because they are primarily trained on natural language rather than low-level control signals. This limitation is especially pronounced for aerial flight, where motion occurs in unconstrained 6-DoF space and small errors in ego-motion can produce large trajectory drift. Generating aerial videos that follow fine-grained inertial actions can support scalable training and evaluation of aerial agents by providing a controllable proxy for real-world or expensive simulation data. To address this problem, we propose \textbf{Aero-World}, a method for converting a pretrained image-to-video diffusion model into a controllable aerial video generator. Aero-World injects sequences of translational acceleration and angular velocity into a pretrained latent diffusion transformer through an action-token stream. A frozen latent-space Physics Probe, trained independently on real video--IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning while avoiding computationally expensive video decoding. We further propose \textbf{AeroBench}, a benchmark for evaluating whether generated drone videos adhere to low-level action signals. AeroBench uses Action Alignment Score (AAS) to measure agreement with commanded inertial actions and Physical Consistency Rate (PCR) to measure temporal motion stability. On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape, with lower FVD (596.5 vs. 1058.6), higher SSIM (0.595 vs. 0.505), and higher Flow-IMU correlation (0.44 vs. 0.20). These results suggest that frozen Physics Probe supervision is a practical mechanism for adapting pretrained video generators toward more action-aligned aerial motion.

Related papers

• UAV Video Deblurring via Motion-Aware Diffusion: A Path to Robust Target Detection [10.233658861023612]
  Unmanned Aerial Vehicles (UAVs) play a crucial role in various scenarios ranging from disaster response to traffic surveillance.<n>Our goal is to explore a computationally-efficient and effective video deblurring approach to enhance UAV target detection performance.
  arXiv  Detail & Related papers  (2026-08-15T14:31:16Z)
• SimWAM: A Simple World Action Model for End-to-End Autonomous Driving [66.93548079427477]
  We present SimWAM, a simple yet effective WAM that leverages future-video prediction as a training-time supervision signal.<n>An isolated attention mask keeps action prediction independent of future frames, allowing trajectory prediction without explicit future-frame generation at inference.<n>Our SimWAM achieves 91.5 PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes.
  arXiv  Detail & Related papers  (2026-08-07T17:59:09Z)
• AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight [42.75555298269899]
  Language-conditioned quadrotor flight requires a policy to ground semantic goals.<n>We present AeroAct, an action-centered world-action model (WAM) for quadrotor navigation.
  arXiv  Detail & Related papers  (2026-07-16T13:46:00Z)
• GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch [66.7936140772745]
  We present GigaWorld-Policy-0.5, an action-centered WAM designed for more efficient robot control.<n>For efficient inference, GigaWorld-Policy-0.5 introduces a Mixture-of-Transformers architecture that separates visual dynamics modeling and action generation into specialized experts.<n>Experiments and ablations show that GigaWorld-Policy-0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control.
  arXiv  Detail & Related papers  (2026-07-15T15:46:42Z)
• Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control [56.828569670698975]
  "Directing the World" is a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control.<n>Our key idea is to decouple control learning while preserving a unified autoregressive video prior.<n>For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy.
  arXiv  Detail & Related papers  (2026-06-26T11:08:09Z)
• EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards [47.255807408091755]
  Video generative models are increasingly used as world models for robotics.<n>Current video world models lack explicit executability constraints.<n>We introduce Executable Video Alignment (EVA), a reinforcement-learning framework for aligning video world models.
  arXiv  Detail & Related papers  (2026-03-18T15:02:19Z)
• Physics-infused Learning for Aerial Manipulator in Winds and Near-Wall Environments [3.895172208362693]
  Aerial manipulation (AM) expands UAV capabilities beyond passive observation to contact-based operations at high altitudes and in otherwise inaccessible environments.<n>This paper integrates a physics-based blade-element model with a learning-based residual force estimator, along with a rotor-speed allocation strategy for disturbance compensation, resulting in a unified control framework.<n>Results demonstrate improved disturbance estimation and trajectory-tracking accuracy over conventional approaches, enabling robust wall-contact execution under challenging aerodynamic conditions.
  arXiv  Detail & Related papers  (2026-03-08T22:08:15Z)
• DiffusionCinema: Text-to-Aerial Cinematography [2.186077977059593]
  Instead of manually piloting the drone, the user simply describes the desired shot.<n>Our system encodes along with an initial visual snapshot from the onboard camera.<n>The generated flight trajectory is then executed autonomously by the UAV.
  arXiv  Detail & Related papers  (2026-01-24T11:12:30Z)
• Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning [15.634803296068009]
  Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity.<n>We study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control.
  arXiv  Detail & Related papers  (2025-12-24T10:00:01Z)
• Robotic VLA Benefits from Joint Learning with Motion Image Diffusion [114.60268819583017]
  Vision-Language-Action (VLA) models have achieved remarkable progress in robotic manipulation by mapping multimodal observations and instructions directly to actions.<n>We propose joint learning with motion image diffusion, a novel strategy that enhances VLA models with motion reasoning capabilities.<n> Experiments in both simulation and real-world environments demonstrate that joint learning with motion image diffusion improves the success rate of pi-series VLAs to 97.5%.
  arXiv  Detail & Related papers  (2025-12-19T19:07:53Z)
• Real-Time Motion-Controllable Autoregressive Video Diffusion [79.32730467857535]
  We propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control.<n>We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement with a trajectory-based reward model.<n>Our design preserves the Markov property through a Self-Rollout learning mechanism and accelerates training by selectively denoising steps.
  arXiv  Detail & Related papers  (2025-10-09T12:17:11Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.