Deep reinforcement learning for guidewire navigation in coronary artery phantom

• URL: http://arxiv.org/abs/2110.01840v1
• Date: Tue, 5 Oct 2021 06:30:03 GMT
• Title: Deep reinforcement learning for guidewire navigation in coronary artery phantom
• Abstract summary: We introduce a deep reinforcement learning framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations. The framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.
• Score: 1.677057841328378
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning(RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. `State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in `stable' phase. Our framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.

Related papers

• Online Joint Calibration of Steering Offset and Planar LiDAR Extrinsics for Wheeled Mobile Robots [0.23624125155742057]
  This paper presents an Extended Kalman Filter (EKF)-based method for online estimation of steering offset and planar LiDAR extrinsics within a bicycle-kinematics model.<n> Experiments on real datasets show that correcting steering offset reduces CTE substantially, validating the effectiveness of the proposed approach.
  arXiv  Detail & Related papers  (2026-08-27T08:19:06Z)
• LineRides: Line-Guided Reinforcement Learning for Bicycle Robot Stunts [11.983924042209447]
  LineRides is a line-guided learning framework that enables a custom bicycle robot to acquire diverse, commandable stunt behaviors.<n>LineRides handles physically infeasible guidelines using a tracking margin that permits controlled deviation.<n>We show that the policy trained with our methods supports seamless transitions between normal driving and stunt execution.
  arXiv  Detail & Related papers  (2026-05-06T16:43:28Z)
• ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigation [57.399685080574756]
  Existing MLLM-based VLN methods rely on imitation learning (IL) and often use DAgger for post-training.<n>We propose ActiveVLN, a VLN framework that explicitly enables active exploration through multi-turn RL.<n>Experiments show that ActiveVLN achieves the largest performance gains over IL baselines compared to both DAgger-based and prior RL-based post-training methods.
  arXiv  Detail & Related papers  (2025-09-16T03:31:46Z)
• SplineFormer: An Explainable Transformer-Based Approach for Autonomous Endovascular Navigation [8.784945912952743]
  Traditional segmentation methods fail to provide accurate real-time shape predictions. We propose SplineFormer, a new transformer-based architecture, designed specifically to predict the continuous, smooth shape of the guidewire. We integrate our SplineFormer into an end-to-end robot navigation system by leveraging the condensed information.
  arXiv  Detail & Related papers  (2025-01-08T14:05:24Z)
• Demystifying the Physics of Deep Reinforcement Learning-Based Autonomous Vehicle Decision-Making [6.243971093896272]
  We use a continuous proximal policy optimization-based DRL algorithm as the baseline model and add a multi-head attention framework in an open-source AV simulation environment. We show that the weights in the first head encode the positions of the neighboring vehicles while the second head focuses on the leader vehicle exclusively.
  arXiv  Detail & Related papers  (2024-03-18T02:59:13Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance [56.184255657175335]
  We develop textbfInferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-20T10:41:03Z)
• PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning [16.397594417992483]
  We propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. In our experiments, PathRL achieves better success rates and reduces angular variability compared to other DRL navigation methods.
  arXiv  Detail & Related papers  (2023-10-20T05:55:13Z)
• End-to-End Reinforcement Learning for Torque Based Variable Height Hopping [5.34772724436823]
  Legged locomotion is arguably the most suited and versatile mode to deal with natural or unstructured terrains. In this paper, we present a end-to-end RL based torque controller that learns to implicitly detect the relevant jump phases. We also extend a method for simulation to reality transfer of the learned controller to contact rich dynamic tasks, resulting in successful deployment on the robot.
  arXiv  Detail & Related papers  (2023-07-31T13:51:29Z)
• Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
  We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
  arXiv  Detail & Related papers  (2023-04-19T17:37:54Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Constrained Reinforcement Learning for Robotics via Scenario-Based Programming [64.07167316957533]
  It is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.
  arXiv  Detail & Related papers  (2022-06-20T07:19:38Z)
• Feedback Linearization of Car Dynamics for Racing via Reinforcement Learning [0.0]
  We seek to learn a linearizing controller to simplify the process of controlling a car to race autonomously. A soft actor-critic approach is used to learn a decoupling matrix and drift vector that effectively correct for errors in a hand-designed linearizing controller. To do so, we posit an extension to the method of learning feedback linearization; a neural network that is trained using supervised learning to convert the output of our linearizing controller to the required input for the racing environment.
  arXiv  Detail & Related papers  (2021-10-20T09:11:18Z)
• Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
  We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
  arXiv  Detail & Related papers  (2021-10-01T15:28:44Z)
• Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation [145.84123197129298]
  Language instruction plays an essential role in the natural language grounded navigation tasks. We exploit to train a more robust navigator which is capable of dynamically extracting crucial factors from the long instruction. Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target.
  arXiv  Detail & Related papers  (2021-07-23T14:11:31Z)
• An A* Curriculum Approach to Reinforcement Learning for RGBD Indoor Robot Navigation [6.660458629649825]
  Recently released photo-realistic simulators such as Habitat allow for the training of networks that output control actions directly from perception. Our paper tries to overcome this problem by separating the training of the perception and control neural nets and increasing the path complexity gradually.
  arXiv  Detail & Related papers  (2021-01-05T20:35:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.