Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics

• URL: http://arxiv.org/abs/2607.25985v1
• Date: Tue, 28 Jul 2026 17:05:47 GMT
• Title: Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics
• Abstract summary: Unmanned aerial vehicles (UAVs) present unique challenges for autonomous control due to their underactuated dynamics.<n>This paper investigates a physics-aware, end-to-end deep reinforcement learning (DRL) approach that acts directly on low-level body inputs.<n>Four DRL algorithms, DDPG, TD3, PPO, and SAC, are evaluated in two stages: (S1) thrust-only hover and (S2) hover with pitch torque and a translated goal.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unmanned aerial vehicles (UAVs), particularly quadcopters, present unique challenges for autonomous control due to their underactuated dynamics: only four available control inputs must govern six degrees of freedom. This paper investigates a physics-aware, end-to-end deep reinforcement learning (DRL) approach that acts directly on low-level body inputs, total thrust and body torques $(T, τ_x, τ_y, τ_z)$, and closes the loop through a high-fidelity Simulink environment. Our simulator integrates a 12-state rigid-body model (MATLAB Level-2 S-Function) with (i) an Action2RPM allocation based on the Moore-Penrose pseudo-inverse of a coefficient matrix derived from thrust and drag terms, and (ii) first-order actuator dynamics for each motor (time constant $T_m = 0.076$ s), including rotor gyroscopic coupling. A shaped reward balances goal-reaching and stability using an exponential position well, attitude penalties, and quadratic velocity costs. Four DRL algorithms, DDPG, TD3, PPO, and SAC, are evaluated in two stages: (S1) thrust-only hover and (S2) hover with pitch torque and a translated goal. Results show that SAC and TD3 achieve superior stability and exploration efficiency, while PPO is less sample-efficient. The study highlights the significance of modeling actuator lags and aerodynamic moments for stable low-level control and provides a reproducible benchmark for quadcopter DRL.

Related papers

• ZiMPedance: Impedance-Aware ZMP Modeling and Control for Payload Carrying with Quadruped Robots [4.183011762144409]
  This paper derives a Zero Moment Point (ZMP) formulation that includes passive payload-interface dynamics, relating stiffness, damping, and payload mass to the stability margin.<n>In simulation, the proposed controller reduces stability violations by up to $10times$, from $7.0%$ to $0.7%$, and increase locomotion efficiency by lowering horizontal ground reaction force effort by up to $15%$ compared to a nominal baseline.
  arXiv  Detail & Related papers  (2026-06-17T10:00:19Z)
• Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls [57.166020875486474]
  We propose textbfAero-World, a method for converting a pretrained image-to-video diffusion model into a controllable aerial video generator.<n>A frozen latent-space Physics Probe, trained independently on real video--IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning.<n>On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape.
  arXiv  Detail & Related papers  (2026-05-19T12:02:17Z)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving [62.43744546817599]
  Vision-language--action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings.<n>We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.<n> MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
  arXiv  Detail & Related papers  (2026-05-13T23:35:14Z)
• Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing [73.88859384645264]
  Out-Of-Distribution (OOD) generalization to various racetracks utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control.<n>This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN)<n>The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model.
  arXiv  Detail & Related papers  (2026-04-10T17:12:07Z)
• Dynamic Control Allocation for Dual-Tilt UAV Platforms [0.6999740786886536]
  This paper focuses on dynamic control allocation for a hexarotor UAV platform.<n>It is assumed that the platform is dual-tilting, meaning that it is able to tilt each propeller independently during flight, along two axis.<n>We present a hierarchical control structure composed of a high-level controller generating the required wrench for the tracking task, and a control allocation law ensuring that the actuators produce such wrench.
  arXiv  Detail & Related papers  (2026-04-07T10:27:47Z)
• CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation [46.950823663585425]
  Continual reinforcement learning requires agents to learn from a sequence of tasks without forgetting previously acquired policies.<n>We introduce a novel benchmark suite for CRL based on realistically simulated robots in the Gazebo simulator.
  arXiv  Detail & Related papers  (2026-02-04T18:54:26Z)
• Reinforcement Learning based 6-DoF Maneuvers for Microgravity Intravehicular Docking: A Simulation Study with Int-Ball2 in ISS-JEM [0.5046831208137847]
  This work presents a reinforcement learning framework for docking of JAXA's Int-Ball2 robot inside a high-fidelity Isaac Sim model of the Japanese Experiment Module (JEM)<n>Using Proximal Policy Optimization (PPO), we train and evaluate controllers under domain-randomized dynamics and bounded observation noise.<n>This enables a controlled study of how Int-Ball2's propulsion physics influence RL-based docking performance in constrained microgravity interiors.
  arXiv  Detail & Related papers  (2025-12-15T16:42:48Z)
• Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes [2.6511518230332753]
  Holonomic autonomous underwater vehicles (AUVs) have the hardware ability for agile maneuvering in both translational and rotational degrees of freedom (DOFs)<n>This paper presents the first general zero-shot sim2real deep reinforcement learning-based (DRL) velocity controller enabling path following and agile 6DOF maneuvering with a training duration of just 3 minutes.
  arXiv  Detail & Related papers  (2025-12-09T14:42:26Z)
• RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking [50.200035833530876]
  RobotDancing is a simple, scalable framework that predicts residual joint targets to explicitly correct dynamics discrepancies.<n>It can track multi-minute, high-energy behaviors (jumps, spins, cartwheels) and deploys zero-shot to hardware with high motion tracking quality.
  arXiv  Detail & Related papers  (2025-09-25T03:30:34Z)
• Explainable AI-Enhanced Supervisory Control for Robust Multi-Agent Robotic Systems [0.0]
  We present an explainable AI-enhanced supervisory control framework for multi-agent robotics.<n>We validated the approach in two contrasting domains, spacecraft formation flying and autonomous underwater vehicles.
  arXiv  Detail & Related papers  (2025-09-18T23:59:13Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.