Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty

• URL: http://arxiv.org/abs/2606.16621v1
• Date: Mon, 15 Jun 2026 12:16:01 GMT
• Title: Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty
• Abstract summary: We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem.<n>We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions.
• Score: 0.560928143828791
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aerial manipulators enable physical interaction in hard-to-reach environments; however, the combined problem of direct whole-body aerial manipulation under rapid arm motion, payload changes, and related unknown dynamic uncertainty remains a largely unsolved problem. We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem. The RL outer loop maps desired 6-degrees-of-freedom (DOF) end-effector targets to coordinated whole-body commands, enabling direct task-driven control without relying on a fully accurate coupled dynamic model in the policy layer. An inner loop then tracks these commands while compensating for transient inertial shifts and uncertainty during execution via a dynamics estimator scheme without requiring system model knowledge. We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions. Compared with RL+PID and RL+INDI+PID baselines, the proposed method reduces end-effector tracking error and improves task success rate across the tested hardware conditions. These results show that combining learned whole-body coordination with estimator-based low-level compensation improves the precision and robustness of aerial manipulation under changing operating conditions.

Related papers

• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• Neural Network-Based Adaptive Event-Triggered Control for Dual-Arm Unmanned Aerial Manipulator Systems [24.187147264441936]
  This paper investigates the control problem of dual-arm unmanned aerial manipulator systems (DAUAMs)<n>Strong coupling between the dual-arm and the multirotor platform, together with unmodeled dynamics and external disturbances, poses significant challenges to stable and accurate operation.<n>An adaptive event-triggered control scheme with neural network-based approximation is proposed to address these issues.
  arXiv  Detail & Related papers  (2026-04-18T16:03:48Z)
• Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances [8.618483849755604]
  A drone equipped with overhead manipulators offer unique capabilities for inspection, maintenance, and contact-based interaction.<n>The motion of the drone and its manipulator is tightly linked, and even small attitude changes caused by wind or control imperfections shift the end-effector away from its intended path.<n>We develop a reinforcement-learning framework with a transformer-based double deep Q learning (DDQN)<n>This allows the controller to anticipate the end-effector's motion through simulated rollouts rather than executing those actions directly on the actual model.
  arXiv  Detail & Related papers  (2026-03-27T17:08:40Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• Safe Whole-Body Loco-Manipulation via Combined Model and Learning-based Control [46.47619556874003]
  We propose a whole-body controller that combines a model-based admittance control for the manipulator arm with a Reinforcement Learning policy for legged locomotion.<n>The admittance controller maps external wrenches--such as those applied by a human during physical interaction--into desired end-effector velocities, allowing for compliant behavior.<n>We validate our approach in both simulation and hardware using the Unitree Go2 quadruped robot with a 6-DoF arm and wrist-mounted 6-DoF Force/Torque sensor.
  arXiv  Detail & Related papers  (2026-03-02T22:31:12Z)
• Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning [15.634803296068009]
  Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity.<n>We study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control.
  arXiv  Detail & Related papers  (2025-12-24T10:00:01Z)
• Custom Non-Linear Model Predictive Control for Obstacle Avoidance in Indoor and Outdoor Environments [0.0]
  This paper introduces a Non-linear Model Predictive Control (NMPC) framework for the DJI Matrice 100. The framework supports various trajectory types and employs a penalty-based cost function for control accuracy in tight maneuvers.
  arXiv  Detail & Related papers  (2024-10-03T17:50:19Z)
• Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
  In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability. We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
  arXiv  Detail & Related papers  (2024-02-04T15:54:03Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• Learning Compliance Adaptation in Contact-Rich Manipulation [81.40695846555955]
  We propose a novel approach for learning predictive models of force profiles required for contact-rich tasks. The approach combines an anomaly detection based on Bidirectional Gated Recurrent Units (Bi-GRU) and an adaptive force/impedance controller.
  arXiv  Detail & Related papers  (2020-05-01T05:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.