Evolutionary Curriculum Training for DRL-Based Navigation Systems
- URL: http://arxiv.org/abs/2306.08870v1
- Date: Thu, 15 Jun 2023 05:56:34 GMT
- Title: Evolutionary Curriculum Training for DRL-Based Navigation Systems
- Authors: Max Asselmeier, Zhaoyi Li, Kelin Yu, Danfei Xu
- Abstract summary: This paper introduces a novel approach called evolutionary curriculum training to tackle collision avoidance challenges.
The primary goal of evolutionary curriculum training is to evaluate the collision avoidance model's competency in various scenarios and create curricula to enhance its skills insufficient.
We benchmark the performance of our model across five structured environments to validate the hypothesis that this evolutionary training environment leads to a higher success rate and a lower average number of collisions.
- Score: 5.8633910194112335
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising
method for robot collision avoidance. However, such DRL models often come with
limitations, such as adapting effectively to structured environments containing
various pedestrians. In order to solve this difficulty, previous research has
attempted a few approaches, including training an end-to-end solution by
integrating a waypoint planner with DRL and developing a multimodal solution to
mitigate the drawbacks of the DRL model. However, these approaches have
encountered several issues, including slow training times, scalability
challenges, and poor coordination among different models. To address these
challenges, this paper introduces a novel approach called evolutionary
curriculum training to tackle these challenges. The primary goal of
evolutionary curriculum training is to evaluate the collision avoidance model's
competency in various scenarios and create curricula to enhance its
insufficient skills. The paper introduces an innovative evaluation technique to
assess the DRL model's performance in navigating structured maps and avoiding
dynamic obstacles. Additionally, an evolutionary training environment generates
all the curriculum to improve the DRL model's inadequate skills tested in the
previous evaluation. We benchmark the performance of our model across five
structured environments to validate the hypothesis that this evolutionary
training environment leads to a higher success rate and a lower average number
of collisions. Further details and results at our project website.
Related papers
- Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning [0.0]
Fine-tuning foundation models via reinforcement learning (RL) has proven promising for aligning to downstream objectives.
We exploit the hierarchical nature of diffusion models (DMs) and train them dynamically at each epoch with a tailored RL method.
We show that models trained with HRF achieve better preservation of diversity in downstream tasks, thus enhancing the fine-tuning robustness and at uncompromising mean rewards.
arXiv Detail & Related papers (2024-10-10T19:06:23Z) - Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning.
Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques.
Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
arXiv Detail & Related papers (2024-05-30T23:20:23Z) - PASTA: Pretrained Action-State Transformer Agents [10.654719072766495]
Self-supervised learning has brought about a revolutionary paradigm shift in various computing domains.
Recent approaches involve pre-training transformer models on vast amounts of unlabeled data.
In reinforcement learning, researchers have recently adapted these approaches, developing models pre-trained on expert trajectories.
arXiv Detail & Related papers (2023-07-20T15:09:06Z) - Deploying Offline Reinforcement Learning with Human Feedback [34.11507483049087]
Reinforcement learning has shown promise for decision-making tasks in real-world applications.
One practical framework involves training parameterized policy models from an offline dataset and deploying them in an online environment.
This approach can be risky since the offline training may not be perfect, leading to poor performance of the RL models that may take dangerous actions.
We propose an alternative framework that involves a human supervising the RL models and providing additional feedback in the online deployment phase.
arXiv Detail & Related papers (2023-03-13T12:13:16Z) - On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement
Learning [45.73223325256312]
We investigate whether internal models learned by modern model-based RL algorithms can be leveraged to solve new, distinctly different tasks faster.
We propose Model-Based Cross-Task Transfer (XTRA), a framework for sample-efficient online RL with scalable pretraining and finetuning of learned world models.
arXiv Detail & Related papers (2022-10-19T17:57:06Z) - Training and Evaluation of Deep Policies using Reinforcement Learning
and Generative Models [67.78935378952146]
GenRL is a framework for solving sequential decision-making problems.
It exploits the combination of reinforcement learning and latent variable generative models.
We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
arXiv Detail & Related papers (2022-04-18T22:02:32Z) - DST: Dynamic Substitute Training for Data-free Black-box Attack [79.61601742693713]
We propose a novel dynamic substitute training attack method to encourage substitute model to learn better and faster from the target model.
We introduce a task-driven graph-based structure information learning constrain to improve the quality of generated training data.
arXiv Detail & Related papers (2022-04-03T02:29:11Z) - Pessimistic Model Selection for Offline Deep Reinforcement Learning [56.282483586473816]
Deep Reinforcement Learning (DRL) has demonstrated great potentials in solving sequential decision making problems in many applications.
One main barrier is the over-fitting issue that leads to poor generalizability of the policy learned by DRL.
We propose a pessimistic model selection (PMS) approach for offline DRL with a theoretical guarantee.
arXiv Detail & Related papers (2021-11-29T06:29:49Z) - Few-shot Quality-Diversity Optimization [50.337225556491774]
Quality-Diversity (QD) optimization has been shown to be effective tools in dealing with deceptive minima and sparse rewards in Reinforcement Learning.
We show that, given examples from a task distribution, information about the paths taken by optimization in parameter space can be leveraged to build a prior population, which when used to initialize QD methods in unseen environments, allows for few-shot adaptation.
Experiments carried in both sparse and dense reward settings using robotic manipulation and navigation benchmarks show that it considerably reduces the number of generations that are required for QD optimization in these environments.
arXiv Detail & Related papers (2021-09-14T17:12:20Z) - Bridging Imagination and Reality for Model-Based Deep Reinforcement
Learning [72.18725551199842]
We propose a novel model-based reinforcement learning algorithm, called BrIdging Reality and Dream (BIRD)
It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories.
We demonstrate that our approach improves sample efficiency of model-based planning, and achieves state-of-the-art performance on challenging visual control benchmarks.
arXiv Detail & Related papers (2020-10-23T03:22:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.