Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing

• URL: http://arxiv.org/abs/2603.08019v1
• Date: Mon, 09 Mar 2026 06:49:41 GMT
• Title: Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing
• Abstract summary: Differentiable-physics-based policy learning has recently demonstrated high sample efficiency and remarkable performance across various tasks.<n>We propose DiffRacing, a novel vector field-augmented differentiable policy learning framework.<n>We show that DiffRacing achieves superior sample efficiency, faster convergence, and robust flight performance.
• Score: 8.010590009724636
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous drone racing in complex environments requires agile, high-speed flight while maintaining reliable obstacle avoidance. Differentiable-physics-based policy learning has recently demonstrated high sample efficiency and remarkable performance across various tasks, including agile drone flight and quadruped locomotion. However, applying such methods to drone racing remains difficult, as key objective like gate traversal are inherently hard to express as smooth, differentiable losses. To address these challenges, we propose DiffRacing, a novel vector field-augmented differentiable policy learning framework. DiffRacing integrates differentiable losses and vector fields into the training process to provide continuous and stable gradient signals, balancing obstacle avoidance and high-speed gate traversal. In addition, a differentiable Delta Action Model compensates for dynamics mismatch, enabling efficient sim-to-real transfer without explicit system identification. Extensive simulation and real-world experiments demonstrate that DiffRacing achieves superior sample efficiency, faster convergence, and robust flight performance, thereby demonstrating that vector fields can augment traditional gradient-based policy learning with a task-specific geometric prior.

Related papers

• Fuzzy Logic Theory-based Adaptive Reward Shaping for Robust Reinforcement Learning (FARS) [2.8055179094637683]
  Reinforcement learning (RL) often struggles in real-world tasks with high-dimensional state spaces and long horizons.<n>This paper presents a fuzzy logic based reward shaping method that integrates human intuition into RL reward design.
  arXiv  Detail & Related papers  (2026-04-17T07:25:55Z)
• Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation [53.750389076941396]
  Diffusion-based visuomotor policies capture multimodal action distributions through iterative denoising, but their high inference latency limits real-time robotic control.<n>We propose Ada3Drift, which learns a training-time drifting field that attracts predicted actions toward expert demonstration modes.<n>Ada3Drift achieves state-of-the-art performance while requiring $10times$ fewer function evaluations than diffusion-based alternatives.
  arXiv  Detail & Related papers  (2026-03-12T14:36:53Z)
• Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles [12.806994271928483]
  We propose a novel vision-based curriculum reinforcement learning framework for training a robust controller capable of addressing unseen obstacles in drone racing.<n>Our end-to-end control policy is implemented as a single network, allowing high-speed flight of quadrotors in environments with variable obstacles.
  arXiv  Detail & Related papers  (2026-02-27T13:56:59Z)
• Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing [22.63301233637327]
  Most reinforcement learning methods for drone racing target fixed, obstacle-free tracks, leaving the generalization to unknown, cluttered environments largely unaddressed.<n>This challenge stems from the need to balance racing speed and collision avoidance, limited feasible space causing policy exploration trapped in local optima during training.<n>We propose a two-phase learning framework: an initial soft-collision training phase that preserves policy exploration for high-speed flight, followed by a hard-collision refinement phase that enforces robust obstacle avoidance.
  arXiv  Detail & Related papers  (2025-12-10T12:02:48Z)
• RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation [13.319344167881383]
  We introduce a dual-stage AV-centric simulation framework that conducts imitation learning pre-training in a data-driven simulator.<n>We then learn fine-tuning in a physics-based simulator to enhance style-level controllability.<n>In the fine-tuning stage, we propose RIFT, a novel group-relative RL fine-tuning strategy.
  arXiv  Detail & Related papers  (2025-05-06T09:12:37Z)
• Effective and Efficient Representation Learning for Flight Trajectories [13.128434340483572]
  We argue that different flight analysis tasks share the same useful features of the trajectory.<n>Flight2Vec is a flight-specific representation learning method to address these challenges.
  arXiv  Detail & Related papers  (2024-12-21T10:59:54Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Biologically Inspired Swarm Dynamic Target Tracking and Obstacle Avoidance [0.0]
  This study proposes a novel artificial intelligence (AI) driven flight computer to track dynamic targets using a distributed drone swarm for military applications. The controller integrates a fuzzy interface, a neural network enabling rapid adaption, predictive capability and multi-agent solving.
  arXiv  Detail & Related papers  (2024-10-15T03:47:09Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [57.278726604424556]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers.<n>Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy.<n>We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds [96.74836678572582]
  We present a learning-based approach that allows rapid online adaptation by incorporating pretrained representations through deep learning. Neural-Fly achieves precise flight control with substantially smaller tracking error than state-of-the-art nonlinear and adaptive controllers.
  arXiv  Detail & Related papers  (2022-05-13T21:55:28Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• Learning High-Speed Flight in the Wild [101.33104268902208]
  We propose an end-to-end approach that can autonomously fly quadrotors through complex natural and man-made environments at high speeds. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments.
  arXiv  Detail & Related papers  (2021-10-11T09:43:11Z)
• Identification and Avoidance of Static and Dynamic Obstacles on Point Cloud for UAVs Navigation [7.14505983271756]
  We introduce a technique to distinguish dynamic obstacles from static ones with only point cloud input. A computationally efficient obstacle avoidance motion planning approach is proposed and it is in line with an improved relative velocity method. The approach is able to avoid both static obstacles and dynamic ones in the same framework.
  arXiv  Detail & Related papers  (2021-05-14T02:44:18Z)
• Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp [119.69304125647785]
  This paper introduces a concise yet powerful method to construct Continuous Transition. Specifically, we propose to synthesize new transitions for training by linearly interpolating the consecutive transitions. To keep the constructed transitions authentic, we also develop a discriminator to guide the construction process automatically.
  arXiv  Detail & Related papers  (2020-11-30T01:20:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.