Vision-Guided MPPI for Agile Drone Racing: Navigating Arbitrary Gate Poses via Neural Signed Distance Fields

• URL: http://arxiv.org/abs/2603.07199v1
• Date: Sat, 07 Mar 2026 13:18:07 GMT
• Title: Vision-Guided MPPI for Agile Drone Racing: Navigating Arbitrary Gate Poses via Neural Signed Distance Fields
• Abstract summary: Drone racing requires tight coupling of perception, planning, and control under extreme agility.<n>Recent approaches rely on precomputed spatial reference trajectories or explicit 6-DoF gate pose estimation.<n>We propose a vision guided optimal control framework that enables reference-free agile flight through arbitrarily placed and oriented gates.
• Score: 8.66703842496129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous drone racing requires the tight coupling of perception, planning, and control under extreme agility. However, recent approaches typically rely on precomputed spatial reference trajectories or explicit 6-DoF gate pose estimation, rendering them brittle to spatial perturbations, unmodeled track changes, and sensor noise. Conversely, end-to-end learning policies frequently overfit to specific track layouts and struggle with zero-shot generalization. To address these fundamental limitations, we propose a fully onboard, vision guided optimal control framework that enables reference-free agile flight through arbitrarily placed and oriented gates. Central to our approach is Gate-SDF, a novel, implicitly learned neural signed distance field. Gate-SDF directly processes raw, noisy depth images to predict a continuous spatial field that provides both collision repulsion and active geometric guidance toward the valid traversal area. We seamlessly integrate this representation into a sampling-based Model Predictive Path Integral (MPPI) controller. By fully exploiting GPU parallelism, the framework evaluates these continuous spatial constraints across thousands of simulated trajectory rollouts simultaneously in real time. Furthermore, our formulation inherently maintains spatial consistency, ensuring robust navigation even under severe visual occlusion during aggressive maneuvers. Extensive simulations and real-world experiments demonstrate that the proposed system achieves high-speed agile flight and successfully navigates unseen tracks subject to severe unmodeled gate displacements and orientation perturbations. Videos are available at https://zhaofangguo.github.io/vision_guided_mppi/

Related papers

• AutoPath: Learning Transferable Goal-Conditioned Stochastic Path Prior for Safe Navigation Without Human Demonstrations [103.61864552260316]
  Real-time navigation in dynamic environments requires collision-free and dynamically feasible motion under limited perception.<n>We formulate navigation as learning a transferable goal-aligned path prior that models a reusable distribution over goal-aligned geometry-consistent local paths conditioned on local observations.<n>We develop a structured prior learning framework that parameterizes local paths using a geometry-aware polar action manifold and incorporates risk-sensitive utility shaping with multi-goal distributional rollouts for stable and safety-aware planning.
  arXiv  Detail & Related papers  (2026-07-13T16:00:41Z)
• FLAP: FOV-Constrained Active Perception Planning for Prior-Map-Free 3D Navigation [11.226202958671502]
  We propose a novel planning framework that directly integrates active perception into trajectory optimization.<n>We introduce an active perception sub-trajectory segment with parametric start-time optimization, mitigating collision risks from late obstacle detection.<n>Our formulation enables active perception during arbitrary 3D maneuvers, extending beyond prior methods designed mainly for horizontal motion.
  arXiv  Detail & Related papers  (2026-06-16T07:40:30Z)
• EXACT-MPPI: Exact Signed-Distance Navigation for Arbitrary-Footprint Robots from Point Clouds via Path Integral Control [11.31921914039084]
  EXACT-MPPI is a framework that maps local navigation directly to motion commands.<n>It embeds an analytic, exact signed-distance evaluator into a Model Predictive Path Integral (MPPI) controller.<n>The same framework deploys on differential-drive, Ackermann, and hybrid-mode platforms.
  arXiv  Detail & Related papers  (2026-05-28T09:23:54Z)
• Perception-Aware Time-Optimal Planning for Quadrotor Waypoint Flight [25.084985306310827]
  This paper introduces a unified time-optimal trajectory optimization framework for vision-based quadrotors.<n>It incorporates perception constraints alongside full nonlinear dynamics, rotor actuation limits, aerodynamic effects, camera field-of-view constraints, and convex geometric gate representations.<n>Experiments demonstrate real-world flight speeds up to 9.8 m/s with 0.07 m average tracking error, and closed-loop success rates improved from 55% to 100%.
  arXiv  Detail & Related papers  (2026-03-04T17:22:16Z)
• Spatially-Aware Adaptive Trajectory Optimization with Controller-Guided Feedback for Autonomous Racing [74.83272587893508]
  We present a framework for autonomous raceline optimization that combines NURBS-based trajectory representation, CMA-ES global trajectory optimization, and controller-guided spatial feedback.<n>In simulation, our approach achieves a 17.38% lap time reduction compared to a controller parametrized with maximum static acceleration.<n>On real hardware, tested with different tire compounds ranging from high to low friction, we obtain a 7.60% lap time improvement without explicitly parametrizing friction.
  arXiv  Detail & Related papers  (2026-02-17T15:10:44Z)
• ONRAP: Occupancy-driven Noise-Resilient Autonomous Path Planning [4.58403099322105]
  We propose a practical, implementation-friendly planner that operates on occupancy grids.<n>It generates ego-centric, kinematically feasible paths that safely navigate through static and dynamic obstacles.<n>We validate our approach in simulation with severe localization and perception noises.
  arXiv  Detail & Related papers  (2026-02-14T03:27:46Z)
• Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing [22.63301233637327]
  Most reinforcement learning methods for drone racing target fixed, obstacle-free tracks, leaving the generalization to unknown, cluttered environments largely unaddressed.<n>This challenge stems from the need to balance racing speed and collision avoidance, limited feasible space causing policy exploration trapped in local optima during training.<n>We propose a two-phase learning framework: an initial soft-collision training phase that preserves policy exploration for high-speed flight, followed by a hard-collision refinement phase that enforces robust obstacle avoidance.
  arXiv  Detail & Related papers  (2025-12-10T12:02:48Z)
• Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction [86.53248703859718]
  We introduce a perceptive locomotion framework that merges terrain sensing, gait regulation, and whole-body control into a single reinforcement learning policy.<n>Experiments conducted on a 31-DoF, 1.65 m humanoid robot demonstrate robust locomotion in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2025-12-08T11:42:41Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Layout Sequence Prediction From Noisy Mobile Modality [53.49649231056857]
  Trajectory prediction plays a vital role in understanding pedestrian movement for applications such as autonomous driving and robotics. Current trajectory prediction models depend on long, complete, and accurately observed sequences from visual modalities. We propose LTrajDiff, a novel approach that treats objects obstructed or out of sight as equally important as those with fully visible trajectories.
  arXiv  Detail & Related papers  (2023-10-09T20:32:49Z)
• Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion [83.88829943619656]
  We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals. Our guided diffusion model allows users to constrain trajectories through target waypoints, speed, and specified social groups. We propose utilizing the value function learned during RL training of the animation controller to guide diffusion to produce trajectories better suited for particular scenarios.
  arXiv  Detail & Related papers  (2023-04-04T15:46:42Z)
• Space Non-cooperative Object Active Tracking with Deep Reinforcement Learning [1.212848031108815]
  We propose an end-to-end active visual tracking method based on DQN algorithm, named as DRLAVT. It can guide the chasing spacecraft approach to arbitrary space non-cooperative target merely relied on color or RGBD images. It significantly outperforms position-based visual servoing baseline algorithm that adopts state-of-the-art 2D monocular tracker, SiamRPN.
  arXiv  Detail & Related papers  (2021-12-18T06:12:24Z)
• Learning High-Speed Flight in the Wild [101.33104268902208]
  We propose an end-to-end approach that can autonomously fly quadrotors through complex natural and man-made environments at high speeds. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments.
  arXiv  Detail & Related papers  (2021-10-11T09:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.