Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2003.13321v2
• Date: Tue, 7 Apr 2020 06:37:13 GMT
• Title: Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning
• Authors: Hannes Hase, Mohammad Farid Azampour, Maria Tirindelli, Magdalini Paschali, Walter Simson, Emad Fatemizadeh and Nassir Navab
• Abstract summary: We introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. When testing our proposed model, we obtained a 82.91% chance of navigating correctly to the sacrum from 165 different starting positions.
• Score: 38.136007056617885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task. Our method is trained and evaluated on an in-house collected data-set of 34 volunteers and when compared to pure RL and supervised learning (SL) techniques, it performs substantially better, which highlights the suitability of RL navigation for US-guided procedures. When testing our proposed model, we obtained a 82.91% chance of navigating correctly to the sacrum from 165 different starting positions on 5 different unseen simulated environments.

Related papers

• Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel [83.7466618084902]
  We introduce a Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale navigational instruction-trajectory pairs. Our experiments demonstrate that after several flywheel rounds, the navigator elevates the performance boundary from 70% to 78% SPL on the classic R2R test set. This process results in a superior generator, evidenced by a SPICE increase from 23.5 to 26.2, better than all previous VLN instruction generation methods.
  arXiv  Detail & Related papers  (2024-12-11T15:32:24Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and Delivery Route Prediction [21.335721424944257]
  We present the first attempt to generalize Reinforcement Learning (RL) to the route prediction task, leading to a novel RL-based framework called DRL4Route. DRL4Route can serve as a plug-and-play component to boost the existing deep learning models. It follows the actor-critic architecture which is equipped with a Generalized Advantage Estimator.
  arXiv  Detail & Related papers  (2023-07-30T14:50:31Z)
• Supervised Pretraining Can Learn In-Context Reinforcement Learning [96.62869749926415]
  In this paper, we study the in-context learning capabilities of transformers in decision-making problems. We introduce and study Decision-Pretrained Transformer (DPT), a supervised pretraining method where the transformer predicts an optimal action. We find that the pretrained transformer can be used to solve a range of RL problems in-context, exhibiting both exploration online and conservatism offline.
  arXiv  Detail & Related papers  (2023-06-26T17:58:50Z)
• Deep Black-Box Reinforcement Learning with Movement Primitives [15.184283143878488]
  We present a new algorithm for deep reinforcement learning (RL) It is based on differentiable trust region layers, a successful on-policy deep RL algorithm. We compare our ERL algorithm to state-of-the-art step-based algorithms in many complex simulated robotic control tasks.
  arXiv  Detail & Related papers  (2022-10-18T06:34:52Z)
• Reward Uncertainty for Exploration in Preference-based Reinforcement Learning [88.34958680436552]
  We present an exploration method specifically for preference-based reinforcement learning algorithms. Our main idea is to design an intrinsic reward by measuring the novelty based on learned reward. Our experiments show that exploration bonus from uncertainty in learned reward improves both feedback- and sample-efficiency of preference-based RL algorithms.
  arXiv  Detail & Related papers  (2022-05-24T23:22:10Z)
• Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning [28.045441768064215]
  Reinforcement learning approaches have shown superior ability in solving sequential decision making problems. In this work, we consider the task of training an RL agent without employing the demonstration data. We propose to incorporate the hindsight experience replay (HER) and curriculum learning (CL) techniques with RL to efficiently learn the optimal navigation policy in the dense crowd.
  arXiv  Detail & Related papers  (2021-10-09T13:18:11Z)
• Cross-Modal Contrastive Learning of Representations for Navigation using Lightweight, Low-Cost Millimeter Wave Radar for Adverse Environmental Conditions [1.9822346227538585]
  We propose the use of single-chip millimeter-wave (mmWave) radar for learning-based autonomous navigation. Because mmWave radar signals are often noisy and sparse, we propose a cross-modal contrastive learning for representation (CM-CLR) method. Our proposed end-to-end deep RL policy with contrastive learning successfully navigated the robot through smoke-filled maze environments.
  arXiv  Detail & Related papers  (2021-01-10T11:21:17Z)
• Learning Dexterous Manipulation from Suboptimal Experts [69.8017067648129]
  Relative Entropy Q-Learning (REQ) is a simple policy algorithm that combines ideas from successful offline and conventional RL algorithms. We show how REQ is also effective for general off-policy RL, offline RL, and RL from demonstrations.
  arXiv  Detail & Related papers  (2020-10-16T18:48:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.