ReProHRL: Towards Multi-Goal Navigation in the Real World using
Hierarchical Agents
- URL: http://arxiv.org/abs/2308.08737v1
- Date: Thu, 17 Aug 2023 02:23:59 GMT
- Title: ReProHRL: Towards Multi-Goal Navigation in the Real World using
Hierarchical Agents
- Authors: Tejaswini Manjunath, Mozhgan Navardi, Prakhar Dixit, Bharat Prakash,
Tinoosh Mohsenin
- Abstract summary: We present Ready for Production Hierarchical RL (ReProHRL) that divides tasks with hierarchical multi-goal navigation guided by reinforcement learning.
We also use object detectors as a pre-processing step to learn multi-goal navigation and transfer it to the real world.
For the real-world implementation and proof of concept demonstration, we deploy the proposed method on a nano-drone named Crazyflie with a front camera.
- Score: 1.3194749469702445
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Robots have been successfully used to perform tasks with high precision. In
real-world environments with sparse rewards and multiple goals, learning is
still a major challenge and Reinforcement Learning (RL) algorithms fail to
learn good policies. Training in simulation environments and then fine-tuning
in the real world is a common approach. However, adapting to the real-world
setting is a challenge. In this paper, we present a method named Ready for
Production Hierarchical RL (ReProHRL) that divides tasks with hierarchical
multi-goal navigation guided by reinforcement learning. We also use object
detectors as a pre-processing step to learn multi-goal navigation and transfer
it to the real world. Empirical results show that the proposed ReProHRL method
outperforms the state-of-the-art baseline in simulation and real-world
environments in terms of both training time and performance. Although both
methods achieve a 100% success rate in a simple environment for single
goal-based navigation, in a more complex environment and multi-goal setting,
the proposed method outperforms the baseline by 18% and 5%, respectively. For
the real-world implementation and proof of concept demonstration, we deploy the
proposed method on a nano-drone named Crazyflie with a front camera to perform
multi-goal navigation experiments.
Related papers
- Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning.
Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques.
Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
arXiv Detail & Related papers (2024-05-30T23:20:23Z) - Imitating Shortest Paths in Simulation Enables Effective Navigation and
Manipulation in the Real World [46.977470141707315]
We show that imitating shortest-path planners in simulation produces agents that can proficiently navigate, explore, and manipulate objects.
This surprising result is enabled by our end-to-end, transformer-based, SPOC architecture, powerful visual encoders paired with extensive image augmentation.
arXiv Detail & Related papers (2023-12-05T18:59:45Z) - NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration [57.15811390835294]
This paper describes how we can train a single unified diffusion policy to handle both goal-directed navigation and goal-agnostic exploration.
We show that this unified policy results in better overall performance when navigating to visually indicated goals in novel environments.
Our experiments, conducted on a real-world mobile robot platform, show effective navigation in unseen environments in comparison with five alternative methods.
arXiv Detail & Related papers (2023-10-11T21:07:14Z) - Sim-to-Real Deep Reinforcement Learning with Manipulators for
Pick-and-place [1.7478203318226313]
When transferring a Deep Reinforcement Learning model from simulation to the real world, the performance could be unsatisfactory.
This paper proposes a self-supervised vision-based DRL method that allows robots to pick and place objects effectively.
arXiv Detail & Related papers (2023-09-17T11:51:18Z) - Reparameterized Policy Learning for Multimodal Trajectory Optimization [61.13228961771765]
We investigate the challenge of parametrizing policies for reinforcement learning in high-dimensional continuous action spaces.
We propose a principled framework that models the continuous RL policy as a generative model of optimal trajectories.
We present a practical model-based RL method, which leverages the multimodal policy parameterization and learned world model.
arXiv Detail & Related papers (2023-07-20T09:05:46Z) - Curricular Subgoals for Inverse Reinforcement Learning [21.038691420095525]
Inverse Reinforcement Learning (IRL) aims to reconstruct the reward function from expert demonstrations to facilitate policy learning.
Existing IRL methods mainly focus on learning global reward functions to minimize the trajectory difference between the imitator and the expert.
We propose a novel Curricular Subgoal-based Inverse Reinforcement Learning framework, that explicitly disentangles one task with several local subgoals to guide agent imitation.
arXiv Detail & Related papers (2023-06-14T04:06:41Z) - Ada-NAV: Adaptive Trajectory Length-Based Sample Efficient Policy Learning for Robotic Navigation [72.24964965882783]
Trajectory length plays a pivotal role in the training process of reinforcement learning algorithms.
We introduce Ada-NAV, a novel adaptive trajectory length scheme to enhance the training sample efficiency of RL algorithms.
We demonstrate through simulated and real-world robotic experiments that Ada-NAV outperforms conventional methods.
arXiv Detail & Related papers (2023-06-09T18:45:15Z) - Navigating to Objects in the Real World [76.1517654037993]
We present a large-scale empirical study of semantic visual navigation methods comparing methods from classical, modular, and end-to-end learning approaches.
We find that modular learning works well in the real world, attaining a 90% success rate.
In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
arXiv Detail & Related papers (2022-12-02T01:10:47Z) - CostNet: An End-to-End Framework for Goal-Directed Reinforcement
Learning [9.432068833600884]
Reinforcement Learning (RL) is a general framework concerned with an agent that seeks to maximize rewards in an environment.
There are two approaches, model-based and model-free reinforcement learning, that show concrete results in several disciplines.
This paper introduces a novel reinforcement learning algorithm for predicting the distance between two states in a Markov Decision Process.
arXiv Detail & Related papers (2022-10-03T21:16:14Z) - Forgetful Experience Replay in Hierarchical Reinforcement Learning from
Demonstrations [55.41644538483948]
In this paper, we propose a combination of approaches that allow the agent to use low-quality demonstrations in complex vision-based environments.
Our proposed goal-oriented structuring of replay buffer allows the agent to automatically highlight sub-goals for solving complex hierarchical tasks in demonstrations.
The solution based on our algorithm beats all the solutions for the famous MineRL competition and allows the agent to mine a diamond in the Minecraft environment.
arXiv Detail & Related papers (2020-06-17T15:38:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.