Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom

• URL: http://arxiv.org/abs/2606.12048v1
• Date: Wed, 10 Jun 2026 13:12:45 GMT
• Title: Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom
• Abstract summary: We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery.<n>We overcome this with a combination of pre-training on 128,000 synthetic point clouds and two novel data augmentation techniques.<n>In real robot experiments, our system localizes targets with the required precision of 0.75mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate.
• Score: 18.991664638461227
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-risk applications in robotics, such as robot-assisted surgery, present unique challenges. These systems must be both highly precise and interpretable in order to be deployed in environments with very low tolerance for error or unsafe exploration. We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery, one of the most common interventions in general surgery. After segmentation of a colorless point cloud from a single camera, target positions for the clips are extracted using spline interpolation, and can then be adjusted by the human operator. The segmentation model is trained on only 60 hand-labeled real point clouds, reflecting data scarcity in the surgical domain. We overcome this with a combination of pre-training on 128,000 synthetic point clouds and two novel data augmentation techniques. The motion of the end-effector to each target is visualized for the operator, satisfying the unique motion constraints of minimally-invasive surgery while ensuring that the robot's actions are verifiable and interpretable. In real robot experiments, our system localizes targets with the required precision of 0.75mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate. We provide insights that are applicable to many other surgical and non-surgical tasks that require identifying and navigating to a precise target. Source code and project page: https://github.com/balazsgyenes/kirurc

Related papers

• Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning [46.22753169557017]
  We introduce the Surgical World-Action Model (Surgical WAM), a unified generative model that predicts future endoscopic observations and executable surgical robot action chunks.<n>Surgical WAM first learns surgical visual dynamics from action-free video and is then fine-tuned on the fixed action-labeled budget.<n>On a suite of four simulated surgical manipulation tasks, video pretraining improves the average success rate from 63.5% to 77.8%, including an absolute gain of 20 percentage points on PegTransfer.
  arXiv  Detail & Related papers  (2026-08-11T17:59:13Z)
• Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics [131.25998619318702]
  We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date.<n> GR00T-H is the first open foundation vision-language-action model for medical robotics.<n>We train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation.
  arXiv  Detail & Related papers  (2026-04-22T19:05:17Z)
• Give me scissors: Collision-Free Dual-Arm Surgical Assistive Robot for Instrument Delivery [4.187693745535412]
  During surgery, scrub nurses are required to frequently deliver surgical instruments to surgeons.<n>Existing research on robotic scrub nurses relies on predefined paths for instrument delivery.<n>We present a collision-free dual-arm surgical assistive robot capable of performing instrument delivery.
  arXiv  Detail & Related papers  (2026-03-03T03:14:59Z)
• Localising under the drape: proprioception in the era of distributed surgical robotic system [12.001086860486906]
  We present a marker-free proprioception method that enables precise localisation of surgical robots under their sterile draping.<n>Our method relies on lightweight stereo-RGB cameras and novel transformer-based deep learning models.<n>It builds on the largest multi-centre spatial robotic surgery dataset to date.
  arXiv  Detail & Related papers  (2025-10-27T16:50:12Z)
• Automated Surgical Skill Assessment in Endoscopic Pituitary Surgery using Real-time Instrument Tracking on a High-fidelity Bench-top Phantom [9.41936397281689]
  Improved surgical skill is generally associated with improved patient outcomes, but assessment is subjective and labour-intensive. A new public dataset is introduced, focusing on simulated surgery, using the nasal phase of endoscopic pituitary surgery as an exemplar. A Multilayer Perceptron achieved 87% accuracy in predicting surgical skill level (novice or expert), with the "ratio of total procedure time to instrument visible time" correlated with higher surgical skill.
  arXiv  Detail & Related papers  (2024-09-25T15:27:44Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots [129.46920552019247]
  We propose the use of a Convolution Neural Network (CNN) to segment the robot hand from an image in an egocentric view. We fine-tuned the Mask-RCNN network for the specific task of segmenting the hand of the humanoid robot Vizzy.
  arXiv  Detail & Related papers  (2021-02-09T10:34:32Z)
• Using Conditional Generative Adversarial Networks to Reduce the Effects of Latency in Robotic Telesurgery [0.0]
  In surgery, any micro-delay can injure a patient severely and in some cases, result in fatality. Current surgical robots use calibrated sensors to measure the position of the arms and tools. In this work we present a purely optical approach that provides a measurement of the tool position in relation to the patient's tissues.
  arXiv  Detail & Related papers  (2020-10-07T13:40:44Z)
• Recurrent and Spiking Modeling of Sparse Surgical Kinematics [0.8458020117487898]
  A growing number of studies have used machine learning to analyze video and kinematic data captured from surgical robots. In this study, we explore the possibility of using only kinematic data to predict surgeons of similar skill levels. We report that it is possible to identify surgical fellows receiving near perfect scores in the simulation exercises based on their motion characteristics alone.
  arXiv  Detail & Related papers  (2020-05-12T15:41:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.