FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation

• URL: http://arxiv.org/abs/2505.20783v1
• Date: Tue, 27 May 2025 06:41:21 GMT
• Title: FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation
• Abstract summary: Path planning is a critical component in autonomous drone operations, enabling safe and efficient navigation through complex environments.<n>Recent advances in foundation models, particularly large language models (LLMs) and vision-language models (VLMs) have opened new opportunities for enhanced perception and intelligent decision-making in robotics.<n>This paper proposes foundation model-guided path planners (FM-Planner) and presents a comprehensive benchmarking study and practical validation for drone path planning.
• Score: 1.280979348722635
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Path planning is a critical component in autonomous drone operations, enabling safe and efficient navigation through complex environments. Recent advances in foundation models, particularly large language models (LLMs) and vision-language models (VLMs), have opened new opportunities for enhanced perception and intelligent decision-making in robotics. However, their practical applicability and effectiveness in global path planning remain relatively unexplored. This paper proposes foundation model-guided path planners (FM-Planner) and presents a comprehensive benchmarking study and practical validation for drone path planning. Specifically, we first systematically evaluate eight representative LLM and VLM approaches using standardized simulation scenarios. To enable effective real-time navigation, we then design an integrated LLM-Vision planner that combines semantic reasoning with visual perception. Furthermore, we deploy and validate the proposed path planner through real-world experiments under multiple configurations. Our findings provide valuable insights into the strengths, limitations, and feasibility of deploying foundation models in real-world drone applications and providing practical implementations in autonomous flight. Project site: https://github.com/NTU-ICG/FM-Planner.

Related papers

• MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning [16.474293563670415]
  Multi-modal Large Language Models (MLLMs) have demonstrated remarkable capabilities in semantic understanding and common sense reasoning.<n>Next-token text prediction objectives may fall short of fulfilling the planning objectives for autonomous vehicles.<n>We propose a reinforcement learning fine-tuning approach, MAGNIFIED, that aligns the MLLM-based driving agent with planning objectives by learning from token-level rewards.
  arXiv  Detail & Related papers  (2026-06-02T16:07:28Z)
• OmniPlanner: Universal Exploration and Inspection Path Planning across Robot Morphologies [13.982169714862648]
  This article presents OmniPlanner, a unified planning framework for autonomous exploration and inspection across aerial, ground, and underwater robots.<n>The method integrates volumetric exploration and viewpoint-based inspection, alongside target reach behaviors within a single modular architecture.<n>The framework is validated through extensive simulation studies and field deployments in underground mines, industrial facilities, forests, submarine bunkers, and structured outdoor environments.
  arXiv  Detail & Related papers  (2026-03-04T17:02:26Z)
• MMH-Planner: Multi-Mode Hybrid Trajectory Planning Method for UAV Efficient Flight Based on Real-Time Spatial Awareness [5.419189180406189]
  This paper proposes a multi-mode hybrid trajectory planning method for UAVs based on real-time environmental awareness.<n>We introduce a goal-oriented spatial awareness method that rapidly assesses flight safety in the upcoming environments.<n>Second, a multi-mode hybrid trajectory planning mechanism is proposed, which can enhance the planning efficiency.
  arXiv  Detail & Related papers  (2026-03-03T07:23:19Z)
• A Unified Experimental Architecture for Informative Path Planning: from Simulation to Deployment with GuadalPlanner [69.43049144653882]
  This paper introduces a unified architecture that decouples high-level decision-making from vehicle-specific control.<n>The proposed architecture is realized through GuadalPlanner, which defines standardized interfaces between planning, sensing, and vehicle execution.
  arXiv  Detail & Related papers  (2026-02-11T10:02:31Z)
• VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigation [16.279434375658457]
  VAMOS is a hierarchical VLA that decouples semantic planning from embodiment grounding.<n>We show VAMOS achieves higher success rates in both indoor and complex outdoor navigation.<n>This model significantly enhances single-robot reliability, achieving 3X higher success rates by rejecting physically infeasible plans.
  arXiv  Detail & Related papers  (2025-10-23T17:59:45Z)
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction [57.56072009935036]
  We introduce a new driving paradigm named Policy World Model (PWM)<n>PWM integrates world modeling and trajectory planning within a unified architecture.<n>Our method matches or exceeds state-of-the-art approaches that rely on multi-view and multi-modal inputs.
  arXiv  Detail & Related papers  (2025-10-22T14:57:51Z)
• Integrating Symbolic RL Planning into a BDI-based Autonomous UAV Framework: System Integration and SIL Validation [3.5966087153300057]
  We propose an extended version of the Autonomous Mission Agents for Drones (AMAD) cognitive multi-agent architecture, enhanced with symbolic reinforcement learning for dynamic mission planning and execution.<n>We validated our framework in a Software-in-the-Loop (SIL) environment structured identically to an intended Hardware-In-the-Loop Simulation (HILS) platform.<n> Experimental results demonstrate stable integration and interoperability of modules, successful transitions between BDI-driven and symbolic RL-driven planning phases, and consistent mission performance.
  arXiv  Detail & Related papers  (2025-08-16T03:27:26Z)
• SimuRA: Towards General Goal-Oriented Agent via Simulative Reasoning Architecture with LLM-Based World Model [88.04128601981145]
  We introduce SimuRA, a goal-oriented architecture for generalized agentic reasoning.<n>modelname overcomes the limitations of autoregressive reasoning by introducing a world model for planning via simulation.<n>World-model-based planning, in particular, shows consistent advantage of up to 124% over autoregressive planning.
  arXiv  Detail & Related papers  (2025-07-31T17:57:20Z)
• VLMPlanner: Integrating Visual Language Models with Motion Planning [18.633637485218802]
  VLMPlanner is a hybrid framework that combines a learning-based real-time planner with a vision-language model (VLM) capable of reasoning over raw images.<n>We develop the Context-Adaptive Inference Gate mechanism that enables the VLM to mimic human driving behavior.
  arXiv  Detail & Related papers  (2025-07-27T16:15:21Z)
• SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models [10.671262416557704]
  Vision Foundation Models (VFMs) offer powerful capabilities for visual understanding and reasoning.<n>We present a zero-shot object goal navigation framework that integrates the perceptual strength of VFMs with a model-based planner.<n>We evaluate our approach on the HM3D dataset using the Habitat simulator and demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-06-04T03:04:54Z)
• Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models [63.765846080050906]
  This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location.<n>Experiments show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-12T12:38:20Z)
• SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation [20.743117921048537]
  SmallPlan is a novel framework leveraging Large Language Models as teacher models to train lightweight Small Language Models (SLMs) for high-level path planning tasks.<n>SLMs are trained in a simulation-powered, interleaved manner with LLM-guided supervised fine-tuning and reinforcement learning.<n>SmallPlan is resource-efficient, making it well-suited for edge-device deployment and advancing practical autonomous robotics.
  arXiv  Detail & Related papers  (2025-05-01T19:44:36Z)
• Dynamic Path Navigation for Motion Agents with LLM Reasoning [69.5875073447454]
  Large Language Models (LLMs) have demonstrated strong generalizable reasoning and planning capabilities.<n>We explore the zero-shot navigation and path generation capabilities of LLMs by constructing a dataset and proposing an evaluation protocol.<n>We demonstrate that, when tasks are well-structured in this manner, modern LLMs exhibit substantial planning proficiency in avoiding obstacles while autonomously refining navigation with the generated motion to reach the target.
  arXiv  Detail & Related papers  (2025-03-10T13:39:09Z)
• Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation [64.84996994779443]
  We propose a novel Affordances-Oriented Planner for continuous vision-language navigation (VLN) task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-07-08T12:52:46Z)
• Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving [2.6855803445552233]
  nuPlan is the world's first real-world autonomous driving dataset and benchmark. The benchmark is designed to test the ability of ML-based planners to handle diverse driving situations. We present a detailed analysis of numerous baselines and investigate gaps between ML-based and traditional methods.
  arXiv  Detail & Related papers  (2024-03-07T01:24:59Z)
• Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities [59.02391344178202]
  Vision foundation models (VFMs) serve as potent building blocks for a wide range of AI applications. The scarcity of comprehensive training data, the need for multi-sensor integration, and the diverse task-specific architectures pose significant obstacles to the development of VFMs. This paper delves into the critical challenge of forging VFMs tailored specifically for autonomous driving, while also outlining future directions.
  arXiv  Detail & Related papers  (2024-01-16T01:57:24Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation [1.2783783498844021]
  A typical SOTA system is composed of four main modules -- mapper, global planner, local planner, and command-tracking controller. We build a robust and safe local planner which is designed to generate a velocity plan to track a coarsely planned path from the global planner. Using our framework, a quadruped robot can autonomously navigate in various complex environments without a collision and generate a smoother command plan compared to the baseline method.
  arXiv  Detail & Related papers  (2022-04-19T04:01:44Z)
• End-to-end Interpretable Neural Motion Planner [78.69295676456085]
  We propose a neural motion planner (NMP) for learning to drive autonomously in complex urban scenarios. We design a holistic model that takes as input raw LIDAR data and a HD map and produces interpretable intermediate representations. We demonstrate the effectiveness of our approach in real-world driving data captured in several cities in North America.
  arXiv  Detail & Related papers  (2021-01-17T14:16:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.