Deterministic and Stochastic Analysis of Deep Reinforcement Learning for Low Dimensional Sensing-based Navigation of Mobile Robots

• URL: http://arxiv.org/abs/2209.06328v1
• Date: Tue, 13 Sep 2022 22:28:26 GMT
• Title: Deterministic and Stochastic Analysis of Deep Reinforcement Learning for Low Dimensional Sensing-based Navigation of Mobile Robots
• Authors: Ricardo B. Grando, Junior C. de Jesus, Victor A. Kich, Alisson H. Kolling, Rodrigo S. Guerra, Paulo L. J. Drews-Jr
• Abstract summary: This paper presents a comparative analysis of two Deep-RL techniques - Deep Deterministic Policy Gradients (DDPG) and Soft Actor-Critic (SAC) We aim to contribute by showing how the neural network architecture influences the learning itself, presenting quantitative results based on the time and distance of aerial mobile robots for each approach.
• Score: 0.41562334038629606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deterministic and Stochastic techniques in Deep Reinforcement Learning (Deep-RL) have become a promising solution to improve motion control and the decision-making tasks for a wide variety of robots. Previous works showed that these Deep-RL algorithms can be applied to perform mapless navigation of mobile robots in general. However, they tend to use simple sensing strategies since it has been shown that they perform poorly with a high dimensional state spaces, such as the ones yielded from image-based sensing. This paper presents a comparative analysis of two Deep-RL techniques - Deep Deterministic Policy Gradients (DDPG) and Soft Actor-Critic (SAC) - when performing tasks of mapless navigation for mobile robots. We aim to contribute by showing how the neural network architecture influences the learning itself, presenting quantitative results based on the time and distance of navigation of aerial mobile robots for each approach. Overall, our analysis of six distinct architectures highlights that the stochastic approach (SAC) better suits with deeper architectures, while the opposite happens with the deterministic approach (DDPG).

Related papers

• Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning [1.3725832537448668]
  The paper details the model of a Ackermann robot and the structure and application of the DDPG algorithm. The results demonstrate that the DDPG algorithm outperforms traditional Deep Q-Network (DQN) and Double Deep Q-Network (DDQN) algorithms in path planning tasks.
  arXiv  Detail & Related papers  (2024-07-18T05:18:59Z)
• Enhanced Low-Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots Using Double Deep Reinforcement Learning Techniques [1.191504645891765]
  We present two distinct approaches aimed at enhancing mapless navigation for a ground-based mobile robot. The research methodology primarily involves a comparative analysis between a Deep-RL strategy grounded in the foundational Deep Q-Network (DQN) algorithm, and an alternative approach based on the Double Deep Q-Network (DDQN) algorithm. The proposed methodology is evaluated in three different real environments, revealing that Double Deep structures significantly enhance the navigation capabilities of mobile robots compared to simple Q structures.
  arXiv  Detail & Related papers  (2023-10-20T20:47:07Z)
• NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration [57.15811390835294]
  This paper describes how we can train a single unified diffusion policy to handle both goal-directed navigation and goal-agnostic exploration. We show that this unified policy results in better overall performance when navigating to visually indicated goals in novel environments. Our experiments, conducted on a real-world mobile robot platform, show effective navigation in unseen environments in comparison with five alternative methods.
  arXiv  Detail & Related papers  (2023-10-11T21:07:14Z)
• Robot path planning using deep reinforcement learning [0.0]
  Reinforcement learning methods offer an alternative to map-free navigation tasks. Deep reinforcement learning agents are implemented for both the obstacle avoidance and the goal-oriented navigation task. An analysis of the changes in the behaviour and performance of the agents caused by modifications in the reward function is conducted.
  arXiv  Detail & Related papers  (2023-02-17T20:08:59Z)
• Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots [0.9175368456179858]
  We present two Deep Reinforcement Learning (Deep-RL) approaches to enhance the problem of mapless navigation for a terrestrial mobile robot. Our methodology focus on comparing a Deep-RL technique based on the Deep Q-Network (DQN) algorithm with a second one based on the Double Deep Q-Network (DDQN) algorithm. By using a low-dimensional sensing structure of learning, we show that it is possible to train an agent to perform navigation-related tasks and obstacle avoidance without using complex sensing information.
  arXiv  Detail & Related papers  (2023-01-26T15:23:59Z)
• Gradient-Based Trajectory Optimization With Learned Dynamics [80.41791191022139]
  We use machine learning techniques to learn a differentiable dynamics model of the system from data. We show that a neural network can model highly nonlinear behaviors accurately for large time horizons. In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car.
  arXiv  Detail & Related papers  (2022-04-09T22:07:34Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Multi-Robot Deep Reinforcement Learning for Mobile Navigation [82.62621210336881]
  We propose a deep reinforcement learning algorithm with hierarchically integrated models (HInt) At training time, HInt learns separate perception and dynamics models, and at test time, HInt integrates the two models in a hierarchical manner and plans actions with the integrated model. Our mobile navigation experiments show that HInt outperforms conventional hierarchical policies and single-source approaches.
  arXiv  Detail & Related papers  (2021-06-24T19:07:40Z)
• XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees [55.9643422180256]
  We present a novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments. Our approach uses deep reinforcement learning-based expert policy that is trained using a sim2real paradigm. We highlight the benefits of our algorithm in simulated environments and navigating a Clearpath Jackal robot among moving pedestrians.
  arXiv  Detail & Related papers  (2021-04-22T01:33:10Z)
• Meta-Gradient Reinforcement Learning with an Objective Discovered Online [54.15180335046361]
  We propose an algorithm based on meta-gradient descent that discovers its own objective, flexibly parameterised by a deep neural network. Because the objective is discovered online, it can adapt to changes over time. On the Atari Learning Environment, the meta-gradient algorithm adapts over time to learn with greater efficiency.
  arXiv  Detail & Related papers  (2020-07-16T16:17:09Z)
• Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware [0.0]
  We propose a neuromorphic approach that combines the energy-efficiency of spiking neural networks with the optimality of deep reinforcement learning (DRL) Our framework consists of a spiking actor network (SAN) and a deep critic network, where the two networks were trained jointly using gradient descent. To evaluate our approach, we deployed the trained SAN on Intel's Loihi neuromorphic processor.
  arXiv  Detail & Related papers  (2020-03-02T19:39:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.