AeroGen: Agentic Drone Autonomy through Single-Shot Structured Prompting & Drone SDK

• URL: http://arxiv.org/abs/2603.14236v1
• Date: Sun, 15 Mar 2026 06:16:02 GMT
• Title: AeroGen: Agentic Drone Autonomy through Single-Shot Structured Prompting & Drone SDK
• Abstract summary: AeroGen is an open-loop framework that enables consistently correct single-shot AI-generated drone control programs.<n>We show that AeroGen generates about 40 lines of AeroD Python code in about 20s per mission, in both real-world and simulations.
• Score: 3.1941554288428193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Designing correct UAV autonomy programs is challenging due to joint navigation, sensing and analytics requirements. While LLMs can generate code, their reliability for safety-critical UAVs remains uncertain. This paper presents AeroGen, an open-loop framework that enables consistently correct single-shot AI-generated drone control programs through structured guardrail prompting and integration with the AeroDaaS drone SDK. AeroGen encodes API descriptions, flight constraints and operational world rules directly into the system context prompt, enabling generic LLMs to produce constraint-aware code from user prompts, with minimal example code. We evaluate AeroGen across a diverse benchmark of 20 navigation tasks and 5 drone missions on urban, farm and inspection environments, using both imperative and declarative user prompts. AeroGen generates about 40 lines of AeroDaaS Python code in about 20s per mission, in both real-world and simulations, showing that structured prompting with a well-defined SDK improves robustness, correctness and deployability of LLM-generated drone autonomy programs.

Related papers

• Harness Robotic OS: A Unified Embodied-Agent Runtime for Closed-Loop Quadruped Inspection [4.737450468943755]
  This paper presents textitHarness Robotic OS (HROS), a unified embodied-agent runtime, and Argos, its realization for residential-community inspection.<n>HROS organizes the system into robot runtime, embodied autonomy skills, cognitive agent, and interaction and operations planes.<n>Experiments in a residential property environment achieved 100% waypoint reachability, outdoor localization error below 10cm, local obstacle-response latency below 200ms, and 99% success in alarm delivery.
  arXiv  Detail & Related papers  (2026-09-10T08:25:46Z)
• Show-Harness: Just a VLM Agent Can Play Robots [53.80887083886227]
  Embodied vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging.<n>We present Show-Harness, an Embodied Harness that enablesVLMs to "play" robots through a compact semantic interface linking intent to action.<n>We further develop GUMI, which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to "play" robots across embodiments without specialized teleoperation hardware.
  arXiv  Detail & Related papers  (2026-09-09T17:53:38Z)
• Generative AI for Safe and Photorealistic Drone Light Shows [51.71264054256171]
  Drone light shows are redefining aerial entertainment, yet their widespread adoption is bottlenecked by labor-intensive, manual animation.<n>We introduce SWAN, an end-to-end pipeline that synthesizes photorealistic, large-scale, and collision-free drone choreographies directly from text prompts.<n>We demonstrate scalability by safely orchestrating simulated 2,000-drone formations and validate physical feasibility on a dense real-world swarm of 49 quadcopters.
  arXiv  Detail & Related papers  (2026-06-24T06:34:44Z)
• AerialClaw: An Open-Source Framework for LLM-Driven Autonomous Aerial Agents [28.465530476682684]
  Unmanned aerial vehicles (UAVs) are increasingly used in inspection, search and rescue, environmental monitoring, and emergency response.<n>This paper presents AerialClaw, an open-source software framework that enables UAVs to operate as decision-making aerial agents.
  arXiv  Detail & Related papers  (2026-06-10T14:34:24Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• VLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operator [1.4878644292213625]
  VLN-Pilot is a framework in which a large Vision-and-Language Model assumes the role of a human pilot for indoor drone navigation.<n>Our framework integrates language-driven semantic understanding with visual perception, enabling context-aware, high-level flight behaviors.
  arXiv  Detail & Related papers  (2026-02-05T11:23:11Z)
• A Universal Large Language Model -- Drone Command and Control Interface [0.0]
  We develop and deploy a cloud based Linux machine hosting an MCP server that supports the Mavlink protocol.<n>We demonstrate flight control of a real unmanned aerial vehicle.
  arXiv  Detail & Related papers  (2026-01-21T21:37:54Z)
• Large Language Models to Enhance Multi-task Drone Operations in Simulated Environments [7.546510363080917]
  We propose a method that integrates a fine-tuned CodeT5 model with the Unreal Engine-based AirSim drone simulator.<n>The proposed method exhibits superior task execution efficiency and command understanding capabilities in simulated environments.
  arXiv  Detail & Related papers  (2026-01-13T10:21:17Z)
• UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios [3.099103925863002]
  UAVBench is an open benchmarking benchmark for UAV flight scenarios generated through large language models (LLMs)<n>We present UAVBench_MCQ, a reasoning-oriented extension containing 50,000 multiple-choice questions spanning ten cognitive and ethical reasoning styles.<n>We evaluate 32 state-of-the-art LLMs, including GPT-5, ChatGPT-4o, Gemini 2.5 Flash, DeepSeek V3, Q3wenwenB, and ERNIE 4.5 300B, and find strong performance in perception and policy reasoning but persistent challenges in ethics-aware and resource-constrained decision-making.
  arXiv  Detail & Related papers  (2025-11-14T12:51:48Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• AirPilot: Interpretable PPO-based DRL Auto-Tuned Nonlinear PID Drone Controller for Robust Autonomous Flights [0.046873264197900916]
  AirPilot is a nonlinear Deep Reinforcement Learning (DRL) - enhanced Proportional Integral Derivative (PID) drone controller.<n>AirPilot controller combines the simplicity and effectiveness of traditional PID control with the adaptability, learning capability, and optimization potential of DRL.<n>Airpilot is capable of reducing the navigation error of the default PX4 PID position controller by 90%, improving effective navigation speed of a fine-tuned PID controller by 21%.
  arXiv  Detail & Related papers  (2024-03-30T00:46:43Z)
• Multi-model fusion for Aerial Vision and Dialog Navigation based on human attention aids [69.98258892165767]
  We present an aerial navigation task for the 2023 ICCV Conversation History. We propose an effective method of fusion training of Human Attention Aided Transformer model (HAA-Transformer) and Human Attention Aided LSTM (HAA-LSTM) models.
  arXiv  Detail & Related papers  (2023-08-27T10:32:52Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.