Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances

• URL: http://arxiv.org/abs/2603.26612v1
• Date: Fri, 27 Mar 2026 17:08:40 GMT
• Title: Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances
• Abstract summary: A drone equipped with overhead manipulators offer unique capabilities for inspection, maintenance, and contact-based interaction.<n>The motion of the drone and its manipulator is tightly linked, and even small attitude changes caused by wind or control imperfections shift the end-effector away from its intended path.<n>We develop a reinforcement-learning framework with a transformer-based double deep Q learning (DDQN)<n>This allows the controller to anticipate the end-effector's motion through simulated rollouts rather than executing those actions directly on the actual model.
• Score: 8.618483849755604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Drones equipped with overhead manipulators offer unique capabilities for inspection, maintenance, and contact-based interaction. However, the motion of the drone and its manipulator is tightly linked, and even small attitude changes caused by wind or control imperfections shift the end-effector away from its intended path. This coupling makes reliable tracking difficult and also limits the direct use of learning-based arm controllers that were originally designed for fixed-base robots. These effects appear consistently in our tests whenever the UAV body experiences drift or rapid attitude corrections. To address this behavior, we develop a reinforcement-learning (RL) framework with a transformer-based double deep Q learning (DDQN), with the core idea of using an adaptive beam-search planner that applies a short-horizon beam search over candidate control sequences using the learned critic as the forward estimator. This allows the controller to anticipate the end-effector's motion through simulated rollouts rather than executing those actions directly on the actual model, realizing a software-in-the-loop (SITL) approach. The lookahead relies on value estimates from a Transformer critic that processes short sequences of states, while a DDQN backbone provides the one-step targets needed to keep the learning process stable. Evaluated on a 3-DoF aerial manipulator under identical training conditions, the proposed meta-adaptive planner shows the strongest overall performance with a 10.2% reward increase, a substantial reduction in mean tracking error (from about 6% to 3%), and a 29.6% improvement in the combined reward-error metric relative to the DDQN baseline. Our method exhibits elevated stability in tracking target tip trajectory (by maintaining 5 cm tracking error) when the drone base exhibits drifts due to external disturbances, as opposed to the fixed-beam and Transformer-only variants.

Related papers

• SonicNudge: Controlled Displacement of Hovering UAVs via Estimator-Controller Coupling [11.293063979196434]
  SonicNudge is an attack primitive that transforms low-level inertial errors into controlled displacement of hovering or slow-moving UAVs.<n>This behavior is especially relevant to UAV tasks that require hovering, station-keeping, slow approach, or precise final alignment.
  arXiv  Detail & Related papers  (2026-08-26T03:03:32Z)
• Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty [0.560928143828791]
  We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem.<n>We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions.
  arXiv  Detail & Related papers  (2026-06-15T12:16:01Z)
• ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptation [63.617951135459016]
  Vision-Language-Action (VLA) models have shown promise for robotic manipulation, yet most existing policies reactively by directly regressing actions from current observations.<n>We propose ELAN4D, an embodiment-centric, 4D-aware training framework that enhances policies with future robot key tracks as predictive-temporal supervision.
  arXiv  Detail & Related papers  (2026-05-28T19:03:30Z)
• Instance-level Visual Active Tracking with Occlusion-Aware Planning [61.982298426203165]
  Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space.<n>VAT faces two key bottlenecks in real-world deployment: confusion from visually similar distractors and severe failure under occlusions.<n>We propose OA-VAT, a unified pipeline with three complementary modules.
  arXiv  Detail & Related papers  (2026-04-23T09:11:50Z)
• Passive iFIR filters for data-driven velocity control in robotics [2.8431237407876258]
  We present a passive, data-driven velocity control method for nonlinear robotic manipulators.<n>Using only three minutes of probing data, a VRFT-based design identifies passive iFIR controllers.
  arXiv  Detail & Related papers  (2026-03-31T15:34:36Z)
• Lightweight 3D LiDAR-Based UAV Tracking: An Adaptive Extended Kalman Filtering Approach [2.3646560222366695]
  This paper presents a lightweight LiDAR-based UAV tracking system incorporating an Adaptive Extended Kalman Filter (AEKF) framework.<n>Our approach effectively addresses the challenges posed by sparse, noisy, and nonuniform point cloud data generated by non-repetitive scanning 3D LiDARs.
  arXiv  Detail & Related papers  (2026-03-10T15:19:34Z)
• PreGME: Prescribed Performance Control of Aerial Manipulators based on Variable-Gain ESO [12.679195185695164]
  We propose a novel prescribed performance motion control framework based on variable-gain extended state observers (ESOs)<n>The method includes variable-gain ESOs for real-time estimation of dynamic coupling and a prescribed performance flight control.<n> Experimental results demonstrate that even under the dynamic coupling caused by rapid robotic arm motion, the proposed method achieves high tracking performance.
  arXiv  Detail & Related papers  (2025-12-28T14:54:06Z)
• Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Reward [85.84943447589511]
  This paper introduces AC3 (Actor-Critic for Continuous Chunks), a novel RL framework that learns to generate high-dimensional, continuous action sequences.<n>To make this learning process stable and data-efficient, AC3 incorporates targeted stabilization mechanisms for both the actor and the critic.
  arXiv  Detail & Related papers  (2025-08-15T01:27:15Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Why Change Your Controller When You Can Change Your Planner: Drag-Aware Trajectory Generation for Quadrotor Systems [10.101847906979435]
  Unmodeled aerodynamic drag forces from carried payloads can lead to catastrophic outcomes. We argue that adapting the trajectory generation component keeping the controller fixed can improve trajectory tracking. Our experiments in both simulation and on the Crazyflie hardware platform show that changing the planner reduces tracking error by as much as 83%.
  arXiv  Detail & Related papers  (2024-01-10T07:00:07Z)
• DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control [62.24301794794304]
  Deep Adaptive Trajectory Tracking (DATT) is a learning-based approach that can precisely track arbitrary, potentially infeasible trajectories in the presence of large disturbances in the real world. DATT significantly outperforms competitive adaptive nonlinear and model predictive controllers for both feasible smooth and infeasible trajectories in unsteady wind fields. It can efficiently run online with an inference time less than 3.2 ms, less than 1/4 of the adaptive nonlinear model predictive control baseline.
  arXiv  Detail & Related papers  (2023-10-13T12:22:31Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Actuator Trajectory Planning for UAVs with Overhead Manipulator using Reinforcement Learning [0.3222802562733786]
  We develop a UAV equipped with a controllable arm with two degrees of freedom to carry out actuation tasks on the fly. Our solution is based on employing a Q-learning method to control the trajectory of the tip of the arm, also called end-effector. Our method achieves 92% accuracy in terms of average displacement error using Q-learning with 15,000 episodes.
  arXiv  Detail & Related papers  (2023-08-24T15:06:23Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.