SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learning

• URL: http://arxiv.org/abs/2409.15651v1
• Date: Tue, 24 Sep 2024 01:27:46 GMT
• Title: SurgIRL: Towards Life-Long Learning for Surgical Automation by Incremental Reinforcement Learning
• Authors: Yun-Jie Ho, Zih-Yun Chiu, Yuheng Zhi, Michael C. Yip,
• Abstract summary: We train surgical automation policies through Surgical Incremental Reinforcement Learning (SurgIRL) SurgIRL aims to (1) acquire new skills by referring to external policies (knowledge) and (2) accumulate and reuse these skills to solve multiple unseen tasks incrementally (incremental learning)
• Score: 21.35087120482373
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Surgical automation holds immense potential to improve the outcome and accessibility of surgery. Recent studies use reinforcement learning to learn policies that automate different surgical tasks. However, these policies are developed independently and are limited in their reusability when the task changes, making it more time-consuming when robots learn to solve multiple tasks. Inspired by how human surgeons build their expertise, we train surgical automation policies through Surgical Incremental Reinforcement Learning (SurgIRL). SurgIRL aims to (1) acquire new skills by referring to external policies (knowledge) and (2) accumulate and reuse these skills to solve multiple unseen tasks incrementally (incremental learning). Our SurgIRL framework includes three major components. We first define an expandable knowledge set containing heterogeneous policies that can be helpful for surgical tasks. Then, we propose Knowledge Inclusive Attention Network with mAximum Coverage Exploration (KIAN-ACE), which improves learning efficiency by maximizing the coverage of the knowledge set during the exploration process. Finally, we develop incremental learning pipelines based on KIAN-ACE to accumulate and reuse learned knowledge and solve multiple surgical tasks sequentially. Our simulation experiments show that KIAN-ACE efficiently learns to automate ten surgical tasks separately or incrementally. We also evaluate our learned policies on the da Vinci Research Kit (dVRK) and demonstrate successful sim-to-real transfers.

Related papers

• SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot [3.487327636814225]
  SurgBox is an agent-driven sandbox framework to enhance cognitive capabilities of surgeons in immersive surgical simulations. In particular, we devise Surgery Copilot, an AI-driven assistant to actively coordinate the surgical information stream and support clinical decision-making.
  arXiv  Detail & Related papers  (2024-12-06T17:07:27Z)
• SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation [58.14969377419633]
  We propose spire, a system that decomposes tasks into smaller learning subproblems and second combines imitation and reinforcement learning to maximize their strengths. We find that spire outperforms prior approaches that integrate imitation learning, reinforcement learning, and planning by 35% to 50% in average task performance.
  arXiv  Detail & Related papers  (2024-10-23T17:42:07Z)
• VS-Assistant: Versatile Surgery Assistant on the Demand of Surgeons [29.783300422432763]
  We propose a Versatile Surgery Assistant (VS-Assistant) that can accurately understand the surgeon's intention. We devise a surgical-Calling Tuning strategy to enable the VS-Assistant to understand surgical intentions.
  arXiv  Detail & Related papers  (2024-05-14T02:05:36Z)
• LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic Surgery [57.358568111574314]
  Patient data privacy often restricts the availability of old data when updating the model. Prior CL studies overlooked two vital problems in the surgical domain. This paper proposes addressing these problems with a multimodal large language model (LLM) and an adaptive weight assignment methodology.
  arXiv  Detail & Related papers  (2024-02-26T15:35:24Z)
• Incremental procedural and sensorimotor learning in cognitive humanoid robots [52.77024349608834]
  This work presents a cognitive agent that can learn procedures incrementally. We show the cognitive functions required in each substage and how adding new functions helps address tasks previously unsolved by the agent. Results show that this approach is capable of solving complex tasks incrementally.
  arXiv  Detail & Related papers  (2023-04-30T22:51:31Z)
• Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot [54.80144694888735]
  We introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm. Our method estimates expert-like behaviors with higher values to facilitate productive interactions. Experiments on $10$ surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-02-20T05:38:54Z)
• Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge [11.007322707874184]
  SimSurgSkill 2021 (hosted as a sub-challenge of EndoVis at MICCAI 2021) aimed to promote and foster work in this endeavor. Competitors were tasked with localizing instruments and predicting surgical skill. Using this publicly available dataset and results as a springboard, future work may enable more efficient training of surgeons with advances in surgical data science.
  arXiv  Detail & Related papers  (2022-12-08T18:14:52Z)
• Quantification of Robotic Surgeries with Vision-Based Deep Learning [45.165919577877695]
  We propose a unified deep learning framework, entitled Roboformer, which operates exclusively on videos recorded during surgery. We validated our framework on four video-based datasets of two commonly-encountered types of steps within minimally-invasive robotic surgeries.
  arXiv  Detail & Related papers  (2022-05-06T06:08:35Z)
• Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention [67.1936055742498]
  We show that multi-task learning can effectively scale reset-free learning schemes to much more complex problems. This work shows the ability to learn dexterous manipulation behaviors in the real world with RL without any human intervention.
  arXiv  Detail & Related papers  (2021-04-22T17:38:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.