Next-Generation LLM for UAV: From Natural Language to Autonomous Flight

• URL: http://arxiv.org/abs/2510.21739v1
• Date: Thu, 02 Oct 2025 04:09:58 GMT
• Title: Next-Generation LLM for UAV: From Natural Language to Autonomous Flight
• Authors: Liangqi Yuan, Chuhao Deng, Dong-Jun Han, Inseok Hwang, Sabine Brunswicker, Christopher G. Brinton,
• Abstract summary: This paper presents a comprehensive demonstration and automation roadmap for integrating Large Language Models into UAV operations.<n>The NeLV system processes natural language instructions to orchestrate short-, medium-, and long-range UAV missions.<n>We demonstrate the system's feasibility through three representative use cases spanning different operational scales.
• Score: 29.20652695322384
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rapid advancement of Large Language Models (LLMs), their capabilities in various automation domains, particularly Unmanned Aerial Vehicle (UAV) operations, have garnered increasing attention. Current research remains predominantly constrained to small-scale UAV applications, with most studies focusing on isolated components such as path planning for toy drones, while lacking comprehensive investigation of medium- and long-range UAV systems in real-world operational contexts. Larger UAV platforms introduce distinct challenges, including stringent requirements for airport-based take-off and landing procedures, adherence to complex regulatory frameworks, and specialized operational capabilities with elevated mission expectations. This position paper presents the Next-Generation LLM for UAV (NeLV) system -- a comprehensive demonstration and automation roadmap for integrating LLMs into multi-scale UAV operations. The NeLV system processes natural language instructions to orchestrate short-, medium-, and long-range UAV missions through five key technical components: (i) LLM-as-Parser for instruction interpretation, (ii) Route Planner for Points of Interest (POI) determination, (iii) Path Planner for waypoint generation, (iv) Control Platform for executable trajectory implementation, and (v) UAV monitoring. We demonstrate the system's feasibility through three representative use cases spanning different operational scales: multi-UAV patrol, multi-POI delivery, and multi-hop relocation. Beyond the current implementation, we establish a five-level automation taxonomy that charts the evolution from current LLM-as-Parser capabilities (Level 1) to fully autonomous LLM-as-Autopilot systems (Level 5), identifying technical prerequisites and research challenges at each stage.

Related papers

• Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial [24.92730472637731]
  Uncrewed Aerial Vehicles (UAVs) are widely deployed across diverse applications due to their mobility and agility.<n>Recent advances in Large Language Models (LLMs) offer a transformative opportunity to enhance UAV intelligence.
  arXiv  Detail & Related papers  (2026-02-23T05:56:43Z)
• AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios [64.51320327698231]
  We introduce AerialMind, the first large-scale RMOT benchmark in UAV scenarios.<n>We develop an innovative semi-automated collaborative agent-based labeling assistant framework.<n>We also propose HawkEyeTrack, a novel method that collaboratively enhances vision-language representation learning.
  arXiv  Detail & Related papers  (2025-11-26T04:44:27Z)
• LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks [57.27815890269697]
  This work focuses on maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under energy constraints.<n>We introduce a Large Language Model (LLM)-guided multi-agent learning approach.<n>Results show that our method outperforms existing baselines in secrecy and energy efficiency.
  arXiv  Detail & Related papers  (2025-07-23T04:22:57Z)
• UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning [39.07541452390107]
  Unmanned Aerial Vehicles (UAVs) are evolving into language-interactive platforms, enabling more intuitive forms of human-drone interaction.<n>We formalize this problem as the Flying-on-a-Word (Flow) task and introduce UAV imitation learning as an effective approach.<n>We present UAV-Flow, the first real-world benchmark for language-conditioned, fine-grained UAV control.
  arXiv  Detail & Related papers  (2025-05-21T16:31:28Z)
• Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap [51.198001060683296]
  Large Language Models (LLMs) offer transformative potential to address transportation challenges.<n>This survey first presents LLM4TR, a novel conceptual framework that systematically categorizes the roles of LLMs in transportation.<n>For each role, our review spans diverse applications, from traffic prediction and autonomous driving to safety analytics and urban mobility optimization.
  arXiv  Detail & Related papers  (2025-03-27T11:56:27Z)
• UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility [33.73170899086857]
  Low-altitude mobility, exemplified by unmanned aerial vehicles (UAVs), has introduced transformative advancements across various domains.<n>This paper explores the integration of large language models (LLMs) and UAVs.<n>It categorizes and analyzes key tasks and application scenarios where UAVs and LLMs converge.
  arXiv  Detail & Related papers  (2025-01-04T17:32:12Z)
• MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation [62.854649499866774]
  Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotics manipulation and navigation.<n>We propose a novel multi-agent LLM framework that distributes high-level planning and low-level control code generation across specialized LLM agents.<n>We evaluate our approach on nine RLBench tasks, including long-horizon tasks, and demonstrate its ability to solve robotics manipulation in a zero-shot setting.
  arXiv  Detail & Related papers  (2024-11-26T17:53:44Z)
• Integrating Large Language Models for UAV Control in Simulated Environments: A Modular Interaction Approach [0.3495246564946556]
  This study explores the application of Large Language Models in UAV control. By enabling UAVs to interpret and respond to natural language commands, LLMs simplify the UAV control and usage. The paper discusses several key areas where LLMs can impact UAV technology, including autonomous decision-making, dynamic mission planning, enhanced situational awareness, and improved safety protocols.
  arXiv  Detail & Related papers  (2024-10-23T06:56:53Z)
• A Survey on Vision-Language-Action Models for Embodied AI [90.99896086619854]
  Embodied AI is widely recognized as a key element of artificial general intelligence.<n>A new category of multimodal models has emerged to address language-conditioned robotic tasks in embodied AI.<n>We present the first survey on vision-language-action models for embodied AI.
  arXiv  Detail & Related papers  (2024-05-23T01:43:54Z)
• Large Language Models for UAVs: Current State and Pathways to the Future [6.85423435360359]
  Unmanned Aerial Vehicles (UAVs) have emerged as a transformative technology across diverse sectors. This work explores the significant potential of integrating UAVs and Large Language Models (LLMs) to propel the development of autonomous systems.
  arXiv  Detail & Related papers  (2024-05-02T21:30:10Z)
• Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities [59.02391344178202]
  Vision foundation models (VFMs) serve as potent building blocks for a wide range of AI applications. The scarcity of comprehensive training data, the need for multi-sensor integration, and the diverse task-specific architectures pose significant obstacles to the development of VFMs. This paper delves into the critical challenge of forging VFMs tailored specifically for autonomous driving, while also outlining future directions.
  arXiv  Detail & Related papers  (2024-01-16T01:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.