E2E-Fly: An Integrated Training-to-Deployment System for End-to-End Quadrotor Autonomy

• URL: http://arxiv.org/abs/2604.12916v1
• Date: Tue, 14 Apr 2026 16:00:22 GMT
• Title: E2E-Fly: An Integrated Training-to-Deployment System for End-to-End Quadrotor Autonomy
• Abstract summary: E2E-Fly is an integrated framework featuring an agile quadrotor platform and a full-stack training, validation, and deployment workflow.<n>We introduce a two-stage validation strategy using sim-to-sim transfer and hardware-in-the-loop testing, and deploy policies onto two physical quadrotor platforms.<n>We demonstrate the effectiveness of our framework for training six end-to-end control tasks and deploy them in the real world.
• Score: 12.44680847653453
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training and transferring learning-based policies for quadrotors from simulation to reality remains challenging due to inefficient visual rendering, physical modeling inaccuracies, unmodeled sensor discrepancies, and the absence of a unified platform integrating differentiable physics learning into end-to-end training. While recent work has demonstrated various end-to-end quadrotor control tasks, few systems provide a systematic, zero-shot transfer pipeline, hindering reproducibility and real-world deployment. To bridge this gap, we introduce E2E-Fly, an integrated framework featuring an agile quadrotor platform coupled with a full-stack training, validation, and deployment workflow. The training framework incorporates a high-performance simulator with support for differentiable physics learning and reinforcement learning, alongside structured reward design tailored to common quadrotor tasks. We further introduce a two-stage validation strategy using sim-to-sim transfer and hardware-in-the-loop testing, and deploy policies onto two physical quadrotor platforms via a dedicated low-level control interface and a comprehensive sim-to-real alignment methodology, encompassing system identification, domain randomization, latency compensation, and noise modeling. To the best of our knowledge, this is the first work to systematically unify differentiable physical learning with training, validation, and real-world deployment for quadrotors. Finally, we demonstrate the effectiveness of our framework for training six end-to-end control tasks and deploy them in the real world.

Related papers

• Stabilization Learning: A Paradigm Transition Bridging Control Theory and Machine Learning [12.657204866483681]
  Stabilization learning is an interdisciplinary paradigm that bridges control theory and machine learning.<n>It takes stability as its primary goal, distinguishing itself from certificate learning, and reinforcement learning.<n>This paper elaborates on the two major categories of stability in stabilization learning, as well as three typical application scenarios.
  arXiv  Detail & Related papers  (2026-06-30T12:19:05Z)
• NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation [9.168410028117089]
  We present a complete training and deployment pipeline for reinforcement learning-based navigation.<n>We disentangle the key factors affecting sim-to-real transfer in reinforcement learning-based navigation, including sensor noise, perception failures, system latency, and control response.<n>We propose perturbation-aware fine-tuning, a post-training adaptation strategy that improves transfer robustness.
  arXiv  Detail & Related papers  (2026-05-15T02:58:25Z)
• A Systematic Post-Train Framework for Video Generation [76.26555417456773]
  Large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content.<n>A significant gap remains between their pretraining performance and real-world deployment requirements due to critical issues such as prompt sensitivity, temporal inconsistency, and prohibitive inference costs.<n>We propose a comprehensive post-training framework that systematically aligns pretrained models with user intentions through four synergistic stages.
  arXiv  Detail & Related papers  (2026-04-28T09:34:51Z)
• Telecom World Models: Unifying Digital Twins, Foundation Models, and Predictive Planning for 6G [60.29349053952935]
  We introduce the Telecom World Model(TWM) concept, an architecture for learned, action-conditioned, uncertainty-aware modeling of telecom system dynamics.<n>We propose a three-layer architecture, comprising a field world model for spatial environment prediction, a control/dynamics world model for action-conditioned Key Performance Indicator (KPI) trajectory prediction, and a telecom foundation model layer for intent translation and orchestration.
  arXiv  Detail & Related papers  (2026-04-08T09:41:58Z)
• VisFly-Lab: Unified Differentiable Framework for First-Order Reinforcement Learning of Quadrotor Control [11.762246555812554]
  First-order reinforcement learning with differentiable simulation is promising for quadrotor control.<n>But practical progress remains fragmented across task-specific settings.<n>We present a unified differentiable framework for multi-task quadrotor control.
  arXiv  Detail & Related papers  (2026-03-22T08:42:26Z)
• From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning [59.88543114325153]
  We introduce the Seeing-to-Experiencing framework to scale the capability of navigation foundation models with reinforcement learning.<n>S2E combines the strengths of pre-training on videos and post-training through RL.<n>We establish a comprehensive end-to-end evaluation benchmark, NavBench-GS, built on photorealistic 3DGS reconstructions of real-world scenes.
  arXiv  Detail & Related papers  (2025-07-29T17:26:10Z)
• A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment [48.90852123901697]
  We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors.<n>Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
  arXiv  Detail & Related papers  (2025-04-21T14:25:23Z)
• Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control [2.5709786140685633]
  This research advances the integration of Soft Actor-Critic with digital twins for industrial robotics applications.<n>The system architecture combines Unity's simulation environment with ROS2 for seamless digital twin synchronization.<n>Results show rapid policy convergence and robust task execution in both simulated and physical environments.
  arXiv  Detail & Related papers  (2025-01-29T22:06:53Z)
• Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features [59.892436892964376]
  We investigate the minimal data requirements and architectural adaptations necessary to achieve robust closed-loop performance with vision-based control policies.<n>Our findings are synthesized in Flex (Fly lexically), a framework that uses pre-trained Vision Language Models (VLMs) as frozen patch-wise feature extractors.<n>We demonstrate the effectiveness of this approach on a quadrotor fly-to-target task, where agents trained via behavior cloning successfully generalize to real-world scenes.
  arXiv  Detail & Related papers  (2024-10-16T19:59:31Z)
• Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policies [0.0]
  This work presents a mixed-reality (MR) digital twin (DT) framework capable of boosting training speeds by selectively scaling parallelized simulation workloads on-demand.<n>We study the effect of: (i) agent and environment parallelization on training time, and (ii) systematic domain randomization on zero-shot sim2real transfer.<n>Results indicate up to 76.3% reduction in training time with the proposed parallelization scheme and sim2real gap as low as 2.9%.
  arXiv  Detail & Related papers  (2024-03-16T18:47:04Z)
• Modular Neural Network Policies for Learning In-Flight Object Catching with a Robot Hand-Arm System [55.94648383147838]
  We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, and (iv) a grasping control policy trained to perform soft catching motions. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight
  arXiv  Detail & Related papers  (2023-12-21T16:20:12Z)
• A Multifidelity Sim-to-Real Pipeline for Verifiable and Compositional Reinforcement Learning [18.42455580047624]
  We propose and demonstrate a compositional framework for training and verifying reinforcement learning systems. By decomposing complex robotic tasks into component subtasks, the framework allows for the independent training and testing of the corresponding subtask policies. In an experimental case study we apply the framework to train and deploy a compositional RL system that successfully pilots a Warthog unmanned ground robot.
  arXiv  Detail & Related papers  (2023-12-02T23:46:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.