A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMs

• URL: http://arxiv.org/abs/2311.12893v1
• Date: Tue, 21 Nov 2023 08:09:00 GMT
• Title: A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMs
• Authors: Jiageng Zhong, Ming Li, Yinliang Chen, Zihang Wei, Fan Yang, Haoran Shen
• Abstract summary: This paper proposes a vision-based planning system that combines tracking and trajectory prediction of dynamic obstacles to achieve efficient and reliable autonomous flight. We conduct experiments in both simulation and real-world environments, and the results indicate that our approach can successfully detect and avoid obstacles in dynamic environments in real-time.
• Score: 6.747468447244154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For intelligent quadcopter UAVs, a robust and reliable autonomous planning system is crucial. Most current trajectory planning methods for UAVs are suitable for static environments but struggle to handle dynamic obstacles, which can pose challenges and even dangers to flight. To address this issue, this paper proposes a vision-based planning system that combines tracking and trajectory prediction of dynamic obstacles to achieve efficient and reliable autonomous flight. We use a lightweight object detection algorithm to identify dynamic obstacles and then use Kalman Filtering to track and estimate their motion states. During the planning phase, we not only consider static obstacles but also account for the potential movements of dynamic obstacles. For trajectory generation, we use a B-spline-based trajectory search algorithm, which is further optimized with various constraints to enhance safety and alignment with the UAV's motion characteristics. We conduct experiments in both simulation and real-world environments, and the results indicate that our approach can successfully detect and avoid obstacles in dynamic environments in real-time, offering greater reliability compared to existing approaches. Furthermore, with the advancements in Natural Language Processing (NLP) technology demonstrating exceptional zero-shot generalization capabilities, more user-friendly human-machine interactions have become feasible, and this study also explores the integration of autonomous planning systems with Large Language Models (LLMs).

Related papers

• Custom Non-Linear Model Predictive Control for Obstacle Avoidance in Indoor and Outdoor Environments [0.0]
  This paper introduces a Non-linear Model Predictive Control (NMPC) framework for the DJI Matrice 100. The framework supports various trajectory types and employs a penalty-based cost function for control accuracy in tight maneuvers.
  arXiv  Detail & Related papers  (2024-10-03T17:50:19Z)
• Hybrid Imitation-Learning Motion Planner for Urban Driving [0.0]
  We propose a novel hybrid motion planner that integrates both learning-based and optimization-based techniques. Our model effectively balances safety and human-likeness, mitigating the trade-off inherent in these objectives. We validate our approach through simulation experiments and further demonstrate its efficacy by deploying it in real-world self-driving vehicles.
  arXiv  Detail & Related papers  (2024-09-04T16:54:31Z)
• OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising [49.86409475232849]
  Trajectory prediction is fundamental in computer vision and autonomous driving. Existing approaches in this field often assume precise and complete observational data. We present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique.
  arXiv  Detail & Related papers  (2024-04-02T18:30:29Z)
• Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding [15.58317292680615]
  We propose a GNN-based approach that uses temporal encoding and imitation learning with data aggregation for learning both the embeddings and the edge prioritization policies. Experiments show that the proposed methods can significantly accelerate online planning over state-of-the-art complete dynamic planning algorithms.
  arXiv  Detail & Related papers  (2022-10-16T01:27:16Z)
• A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera [7.77809394151497]
  We propose a real-time dynamic obstacle tracking and mapping system for quadcopter obstacle avoidance using an RGB-D camera. Our methods can successfully track and represent obstacles in dynamic environments in real-time and safely avoid obstacles.
  arXiv  Detail & Related papers  (2022-09-17T05:32:33Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• DPMPC-Planner: A real-time UAV trajectory planning framework for complex static environments with dynamic obstacles [0.9462808515258462]
  Safe UAV navigation is challenging due to the complex environment structures, dynamic obstacles, and uncertainties from measurement noises and unpredictable moving obstacle behaviors. This paper proposes a trajectory planning framework to achieve safe navigation considering complex static environments with dynamic obstacles.
  arXiv  Detail & Related papers  (2021-09-14T23:51:02Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications [69.24726496448713]
  We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV. We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
  arXiv  Detail & Related papers  (2021-05-28T08:45:37Z)
• MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control [46.86174832000696]
  Reasoning about human motion is a core component of modern human-robot interactive systems. One of the main uses of behavior prediction in autonomous systems is to inform robot motion planning and control. We propose a new output representation for trajectory forecasting that is more amenable to downstream planning and control use.
  arXiv  Detail & Related papers  (2020-09-16T07:32:37Z)
• Learning to Track Dynamic Targets in Partially Known Environments [48.49957897251128]
  We use a deep reinforcement learning approach to solve active target tracking. In particular, we introduce Active Tracking Target Network (ATTN), a unified RL policy that is capable of solving major sub-tasks of active target tracking.
  arXiv  Detail & Related papers  (2020-06-17T22:45:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.