Two-step dynamic obstacle avoidance
- URL: http://arxiv.org/abs/2311.16841v1
- Date: Tue, 28 Nov 2023 14:55:50 GMT
- Title: Two-step dynamic obstacle avoidance
- Authors: Fabian Hart, Martin Waltz, Ostap Okhrin
- Abstract summary: This paper proposes a two-step architecture for handling Dynamic obstacle avoidance tasks by combining supervised and reinforcement learning.
In the first step, we introduce a data-driven approach to estimate the collision risk of an obstacle using a recurrent neural network.
In the second step, we include these collision risk estimates into the observation space of an RL agent to increase its situational awareness.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Dynamic obstacle avoidance (DOA) is a fundamental challenge for any
autonomous vehicle, independent of whether it operates in sea, air, or land.
This paper proposes a two-step architecture for handling DOA tasks by combining
supervised and reinforcement learning (RL). In the first step, we introduce a
data-driven approach to estimate the collision risk of an obstacle using a
recurrent neural network, which is trained in a supervised fashion and offers
robustness to non-linear obstacle movements. In the second step, we include
these collision risk estimates into the observation space of an RL agent to
increase its situational awareness.~We illustrate the power of our two-step
approach by training different RL agents in a challenging environment that
requires to navigate amid multiple obstacles. The non-linear movements of
obstacles are exemplarily modeled based on stochastic processes and periodic
patterns, although our architecture is suitable for any obstacle dynamics. The
experiments reveal that integrating our collision risk metrics into the
observation space doubles the performance in terms of reward, which is
equivalent to halving the number of collisions in the considered environment.
Furthermore, we show that the architecture's performance improvement is
independent of the applied RL algorithm.
Related papers
- Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability.
We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
arXiv Detail & Related papers (2024-02-04T15:54:03Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - Reinforcement Learning with Human Feedback for Realistic Traffic
Simulation [53.85002640149283]
Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge.
This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
arXiv Detail & Related papers (2023-09-01T19:29:53Z) - Seeing is not Believing: Robust Reinforcement Learning against Spurious
Correlation [57.351098530477124]
We consider one critical type of robustness against spurious correlation, where different portions of the state do not have correlations induced by unobserved confounders.
A model that learns such useless or even harmful correlation could catastrophically fail when the confounder in the test case deviates from the training one.
Existing robust algorithms that assume simple and unstructured uncertainty sets are therefore inadequate to address this challenge.
arXiv Detail & Related papers (2023-07-15T23:53:37Z) - Sequential Neural Barriers for Scalable Dynamic Obstacle Avoidance [7.375976854181687]
We propose a novel method for compositional learning of Sequential Neural Control Barrier models (SNCBFs)
Our approach exploits an important observation: the spatial interaction patterns of multiple dynamic obstacles can be decomposed and predicted through temporal sequences of states for each obstacle.
We demonstrate the benefits of the proposed methods in improving dynamic collision avoidance in comparison with existing methods.
arXiv Detail & Related papers (2023-07-06T14:24:17Z) - Continuous Trajectory Generation Based on Two-Stage GAN [50.55181727145379]
We propose a novel two-stage generative adversarial framework to generate the continuous trajectory on the road network.
Specifically, we build the generator under the human mobility hypothesis of the A* algorithm to learn the human mobility behavior.
For the discriminator, we combine the sequential reward with the mobility yaw reward to enhance the effectiveness of the generator.
arXiv Detail & Related papers (2023-01-16T09:54:02Z) - Enhanced method for reinforcement learning based dynamic obstacle
avoidance by assessment of collision risk [0.0]
This paper proposes a general training environment where we gain control over the difficulty of the obstacle avoidance task.
We found that shifting the training towards a greater task difficulty can massively increase the final performance.
arXiv Detail & Related papers (2022-12-08T07:46:42Z) - Smooth Trajectory Collision Avoidance through Deep Reinforcement
Learning [0.0]
We propose several novel agent state and reward function designs to tackle two critical issues in DRL-based navigation solutions.
Our model relies on margin reward and smoothness constraints to ensure UAVs fly smoothly while greatly reducing the chance of collision.
arXiv Detail & Related papers (2022-10-12T16:27:32Z) - Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation [78.17108227614928]
We propose a benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation.
We consider a value-based and policy-gradient Deep Reinforcement Learning (DRL)
We also propose a verification strategy that checks the behavior of the trained models over a set of desired properties.
arXiv Detail & Related papers (2021-12-16T16:53:56Z) - Congestion-aware Multi-agent Trajectory Prediction for Collision
Avoidance [110.63037190641414]
We propose to learn congestion patterns explicitly and devise a novel "Sense--Learn--Reason--Predict" framework.
By decomposing the learning phases into two stages, a "student" can learn contextual cues from a "teacher" while generating collision-free trajectories.
In experiments, we demonstrate that the proposed model is able to generate collision-free trajectory predictions in a synthetic dataset.
arXiv Detail & Related papers (2021-03-26T02:42:33Z) - COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using
Deep Reinforcement Learning [0.0]
Path Following and Collision Avoidance, be it for unmanned surface vessels or other autonomous vehicles, are two fundamental guidance problems in robotics.
In this article, we explore the potential of Proximal Policy Optimization (PPO), a DRL algorithm with demonstrated state-of-the-art performance on Continuous Control tasks.
Based on high-fidelity elevation and AIS tracking data from the Trondheim Fjord, an inlet of the Norwegian sea, we evaluate the trained agent's performance in challenging, dynamic real-world scenarios.
arXiv Detail & Related papers (2020-06-16T22:05:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.