Actuator Trajectory Planning for UAVs with Overhead Manipulator using
Reinforcement Learning
- URL: http://arxiv.org/abs/2308.12843v2
- Date: Fri, 25 Aug 2023 16:28:12 GMT
- Title: Actuator Trajectory Planning for UAVs with Overhead Manipulator using
Reinforcement Learning
- Authors: Hazim Alzorgan, Abolfazl Razi, Ata Jahangir Moshayedi
- Abstract summary: We develop a UAV equipped with a controllable arm with two degrees of freedom to carry out actuation tasks on the fly.
Our solution is based on employing a Q-learning method to control the trajectory of the tip of the arm, also called end-effector.
Our method achieves 92% accuracy in terms of average displacement error using Q-learning with 15,000 episodes.
- Score: 0.3222802562733786
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we investigate the operation of an aerial manipulator system,
namely an Unmanned Aerial Vehicle (UAV) equipped with a controllable arm with
two degrees of freedom to carry out actuation tasks on the fly. Our solution is
based on employing a Q-learning method to control the trajectory of the tip of
the arm, also called end-effector. More specifically, we develop a motion
planning model based on Time To Collision (TTC), which enables a quadrotor UAV
to navigate around obstacles while ensuring the manipulator's reachability.
Additionally, we utilize a model-based Q-learning model to independently track
and control the desired trajectory of the manipulator's end-effector, given an
arbitrary baseline trajectory for the UAV platform. Such a combination enables
a variety of actuation tasks such as high-altitude welding, structural
monitoring and repair, battery replacement, gutter cleaning, skyscrapper
cleaning, and power line maintenance in hard-to-reach and risky environments
while retaining compatibility with flight control firmware. Our RL-based
control mechanism results in a robust control strategy that can handle
uncertainties in the motion of the UAV, offering promising performance.
Specifically, our method achieves 92% accuracy in terms of average displacement
error (i.e. the mean distance between the target and obtained trajectory
points) using Q-learning with 15,000 episodes
Related papers
- Reaching the Limit in Autonomous Racing: Optimal Control versus
Reinforcement Learning [66.10854214036605]
A central question in robotics is how to design a control system for an agile mobile robot.
We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting.
Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour.
arXiv Detail & Related papers (2023-10-17T02:40:27Z) - Toward collision-free trajectory for autonomous and pilot-controlled
unmanned aerial vehicles [1.018017727755629]
This study makes greater use of electronic conspicuity (EC) information made available by PilotAware Ltd in developing an advanced collision management methodology.
The merits of the DACM methodology have been demonstrated through extensive simulations and real-world field tests in avoiding mid-air collisions.
arXiv Detail & Related papers (2023-09-18T18:24:31Z) - Convergence of Communications, Control, and Machine Learning for Secure
and Autonomous Vehicle Navigation [78.60496411542549]
Connected and autonomous vehicles (CAVs) can reduce human errors in traffic accidents, increase road efficiency, and execute various tasks. Reaping these benefits requires CAVs to autonomously navigate to target destinations.
This article proposes solutions using the convergence of communication theory, control theory, and machine learning to enable effective and secure CAV navigation.
arXiv Detail & Related papers (2023-07-05T21:38:36Z) - Designing a Robust Low-Level Agnostic Controller for a Quadrotor with
Actor-Critic Reinforcement Learning [0.38073142980732994]
We introduce domain randomization during the training phase of a low-level waypoint guidance controller based on Soft Actor-Critic.
We show that, by introducing a certain degree of uncertainty in quadrotor dynamics during training, we can obtain a controller that is capable to perform the proposed task using a larger variation of quadrotor parameters.
arXiv Detail & Related papers (2022-10-06T14:58:19Z) - Obstacle Avoidance for UAS in Continuous Action Space Using Deep
Reinforcement Learning [9.891207216312937]
Obstacle avoidance for small unmanned aircraft is vital for the safety of future urban air mobility.
We propose a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) to guide autonomous UAS to their destinations.
Results show that the proposed model can provide accurate and robust guidance and resolve conflict with a success rate of over 99%.
arXiv Detail & Related papers (2021-11-13T04:44:53Z) - Data-Efficient Deep Reinforcement Learning for Attitude Control of
Fixed-Wing UAVs: Field Experiments [0.37798600249187286]
We show that DRL can successfully learn to perform attitude control of a fixed-wing UAV operating directly on the original nonlinear dynamics.
We deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller.
arXiv Detail & Related papers (2021-11-07T19:07:46Z) - OSCAR: Data-Driven Operational Space Control for Adaptive and Robust
Robot Manipulation [50.59541802645156]
Operational Space Control (OSC) has been used as an effective task-space controller for manipulation.
We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors.
We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
arXiv Detail & Related papers (2021-10-02T01:21:38Z) - Towards Safe Control of Continuum Manipulator Using Shielded Multiagent
Reinforcement Learning [1.2647816797166165]
The control of the robot is formulated as a one-DoF, one agent problem in the MADQN framework to improve the learning efficiency.
Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads.
arXiv Detail & Related papers (2021-06-15T05:55:05Z) - 3D UAV Trajectory and Data Collection Optimisation via Deep
Reinforcement Learning [75.78929539923749]
Unmanned aerial vehicles (UAVs) are now beginning to be deployed for enhancing the network performance and coverage in wireless communication.
It is challenging to obtain an optimal resource allocation scheme for the UAV-assisted Internet of Things (IoT)
In this paper, we design a new UAV-assisted IoT systems relying on the shortest flight path of the UAVs while maximising the amount of data collected from IoT devices.
arXiv Detail & Related papers (2021-06-06T14:08:41Z) - Transferable Deep Reinforcement Learning Framework for Autonomous
Vehicles with Joint Radar-Data Communications [69.24726496448713]
We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions.
We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV.
We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
arXiv Detail & Related papers (2021-05-28T08:45:37Z) - Efficient UAV Trajectory-Planning using Economic Reinforcement Learning [65.91405908268662]
We introduce REPlanner, a novel reinforcement learning algorithm inspired by economic transactions to distribute tasks between UAVs.
We formulate the path planning problem as a multi-agent economic game, where agents can cooperate and compete for resources.
As the system computes task distributions via UAV cooperation, it is highly resilient to any change in the swarm size.
arXiv Detail & Related papers (2021-03-03T20:54:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.