XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface

• URL: http://arxiv.org/abs/2511.20254v1
• Date: Tue, 25 Nov 2025 12:29:10 GMT
• Title: XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface
• Authors: Alexander C. Jenke, Gregor Just, Claas de Boer, Martin Wagner, Sebastian Bodenstedt, Stefanie Speidel,
• Abstract summary: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback.<n>Da Vinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm.<n>We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile.
• Score: 36.952464191650435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.

Related papers

• Neuromorphic Synergy for Video Binarization [54.195375576583864]
  Bimodal objects serve as a visual form to embed information that can be easily recognized by vision systems. Neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first de-blur and then binarize the images in a real-time manner. We propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space. We also develop an efficient integration method to propagate this binary image to high frame rate binary video.
  arXiv  Detail & Related papers  (2024-02-20T01:43:51Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• Deep Learning Computer Vision Algorithms for Real-time UAVs On-board Camera Image Processing [77.34726150561087]
  This paper describes how advanced deep learning based computer vision algorithms are applied to enable real-time on-board sensor processing for small UAVs. All algorithms have been developed using state-of-the-art image processing methods based on deep neural networks.
  arXiv  Detail & Related papers  (2022-11-02T11:10:42Z)
• AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomy [42.20922574566824]
  We present and release the first integrated dataset with multiple image-based perception tasks to facilitate learning-based automation in hysterectomy surgery. Our AutoLaparo dataset is developed based on full-length videos of entire hysterectomy procedures. Specifically, three different yet highly correlated tasks are formulated in the dataset, including surgical workflow recognition, laparoscope motion prediction, and instrument and key anatomy segmentation.
  arXiv  Detail & Related papers  (2022-08-03T13:17:23Z)
• CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification [53.56290185900837]
  We propose an algorithm for optimizing a neural network for object detection using preliminary binary frame classification. An autonomous mobile robot with 6 rolling-shutter cameras on the perimeter providing a 360-degree field of view was used as the experimental setup.
  arXiv  Detail & Related papers  (2021-10-22T15:05:37Z)
• Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction [6.56651216023737]
  We introduce a method that allows to extract a laparoscope holder's actions from videos of laparoscopic interventions. We synthetically add camera motion to a newly acquired dataset of camera motion free da Vinci surgery image sequences. We find our method transfers from our camera motion free da Vinci surgery dataset to videos of laparoscopic interventions, outperforming classical homography estimation approaches in both, precision by 41%, and runtime on a CPU by 43%.
  arXiv  Detail & Related papers  (2021-09-30T13:05:37Z)
• Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging [1.603589863010401]
  Activity recognition computer vision algorithms can be used to detect the presence of autism-related behaviors. We document the advantages and limitations of current feature representation techniques for activity recognition when applied to head banging detection. We create a computer vision classifier for detecting head banging in home videos using a time-distributed convolutional neural network.
  arXiv  Detail & Related papers  (2021-01-10T05:37:05Z)
• Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection [30.042965489804356]
  We propose a solution to the tool detection and localization open problem in RAS video understanding. We propose an architecture using multimodal convolutional neural networks for fast detection and localization of tools in RAS videos. Our results with an Average Precision (AP) of 91% and a mean time of 0.1 seconds per test frame detection indicate that our study is superior to conventionally used methods for medical imaging.
  arXiv  Detail & Related papers  (2020-07-29T10:59:15Z)
• Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery [1.1033115844630357]
  We investigate automatic surgical activity recognition in robot-assisted operations. We collect the first large-scale dataset including 400 full-length multi-perspective videos. We densely annotate the videos with 10 most recognized and clinically relevant classes of activities.
  arXiv  Detail & Related papers  (2020-06-29T16:30:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.