Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review

• URL: http://arxiv.org/abs/2502.14886v1
• Date: Sun, 16 Feb 2025 07:27:20 GMT
• Title: Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review
• Authors: Ufaq Khan, Umair Nawaz, Adnan Qayyum, Shazad Ashraf, Muhammad Bilal, Junaid Qadir,
• Abstract summary: Recent advancements in machine learning (ML) and deep learning (DL) have significantly enhanced surgical scene understanding within minimally invasive surgery (MIS)<n>This paper surveys the integration of state-of-the-art ML and DL technologies, including CNNs, Vision Transformers (ViTs), and foundational models like the Segment Anything Model (SAM)<n>The paper explores the challenges these technologies face, such as data variability and computational demands, and discusses ethical considerations and integration hurdles in clinical settings.
• Score: 3.552525722519539
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in machine learning (ML) and deep learning (DL), particularly through the introduction of foundational models (FMs), have significantly enhanced surgical scene understanding within minimally invasive surgery (MIS). This paper surveys the integration of state-of-the-art ML and DL technologies, including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and foundational models like the Segment Anything Model (SAM), into surgical workflows. These technologies improve segmentation accuracy, instrument tracking, and phase recognition in surgical endoscopic video analysis. The paper explores the challenges these technologies face, such as data variability and computational demands, and discusses ethical considerations and integration hurdles in clinical settings. Highlighting the roles of FMs, we bridge the technological capabilities with clinical needs and outline future research directions to enhance the adaptability, efficiency, and ethical alignment of AI applications in surgery. Our findings suggest that substantial progress has been made; however, more focused efforts are required to achieve seamless integration of these technologies into clinical workflows, ensuring they complement surgical practice by enhancing precision, reducing risks, and optimizing patient outcomes.

Related papers

• Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutions [27.68772584578631]
  The integration of Embodied Intelligence into robotic systems signifies a paradigm shift. Data-driven approaches, advanced computer vision, medical image analysis, and machine learning techniques, are at the forefront of this evolution. We discuss recent advancements in intelligent perception and data-driven control, and their practical applications in robot-assisted procedures.
  arXiv  Detail & Related papers  (2025-04-21T13:49:30Z)
• Scalable Evaluation Framework for Foundation Models in Musculoskeletal MRI Bridging Computational Innovation with Clinical Utility [0.0]
  This study introduces an evaluation framework for assessing the clinical impact and translatability of SAM, MedSAM, and SAM2.<n>We tested these models across zero-shot and finetuned paradigms to assess their ability to process diverse anatomical structures and effectuate clinically reliable biomarkers.
  arXiv  Detail & Related papers  (2025-01-23T04:41:20Z)
• Deep Learning for Surgical Instrument Recognition and Segmentation in Robotic-Assisted Surgeries: A Systematic Review [0.24342814271497581]
  Applying deep learning (DL) for annotating surgical instruments in robot-assisted minimally invasive surgeries represents a significant advancement in surgical technology. These sophisticated DL models have shown notable improvements in the precision and efficiency of detecting and segmenting surgical tools. The application of DL in surgical education is transformative.
  arXiv  Detail & Related papers  (2024-10-09T04:07:38Z)
• Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery [50.3022015601057]
  We propose a predictive neural network that is capable of understanding and predicting critical interactive aspects of surgical workflow from intra-abdominal video. We verify our approach on established surgical datasets and applications, including the detection and prediction of action triplets. Our results demonstrate the superiority of our approach compared to unstructured alternatives.
  arXiv  Detail & Related papers  (2024-02-03T00:58:05Z)
• Prediction of Post-Operative Renal and Pulmonary Complications Using Transformers [69.81176740997175]
  We evaluate the performance of transformer-based models in predicting postoperative acute renal failure, pulmonary complications, and postoperative in-hospital mortality. Our results demonstrate that transformer-based models can achieve superior performance in predicting postoperative complications and outperform traditional machine learning models.
  arXiv  Detail & Related papers  (2023-06-01T14:08:05Z)
• Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing [88.99939660183881]
  We propose a framework for autonomous robotic navigation for subretinal injection. Our method consists of an instrument pose estimation method, an online registration between the robotic and the i OCT system, and trajectory planning tailored for navigation to an injection target. Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method.
  arXiv  Detail & Related papers  (2023-01-17T21:41:21Z)
• Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
  Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community. The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
  arXiv  Detail & Related papers  (2022-07-01T14:17:11Z)
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition [66.51610049869393]
  This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
  arXiv  Detail & Related papers  (2022-04-10T18:51:55Z)
• Generational Frameshifts in Technology: Computer Science and Neurosurgery, The VR Use Case [0.0]
  The democratization of neurosurgery is at hand and will be driven by our development, extraction, and adoption of these tools of the modern world. The ability to perform surgery more safely and more efficiently while capturing the operative details and parsing each component of the operation will open an entirely new epoch advancing our field and all surgical specialties.
  arXiv  Detail & Related papers  (2021-10-08T20:02:17Z)
• Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures [70.69948035469467]
  We take advantage of the latest computer vision methodologies for generating 3D graphs from camera views. We then introduce the Multimodal Semantic Graph Scene (MSSG) which aims at providing unified symbolic and semantic representation of surgical procedures.
  arXiv  Detail & Related papers  (2021-06-09T14:35:44Z)
• Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge [9.986124942784969]
  This work seeks to explore the potential for visual domain adaptation in surgery to overcome data privacy concerns. In particular, we propose to use video from virtual reality (VR) simulations of surgical exercises to develop algorithms to recognize tasks in a clinical-like setting. We present the performance of the different approaches to solve visual domain adaptation developed by challenge participants.
  arXiv  Detail & Related papers  (2021-02-26T18:45:28Z)
• Machine Learning in Nano-Scale Biomedical Engineering [77.75587007080894]
  We review the existing research regarding the use of machine learning in nano-scale biomedical engineering. The main challenges that can be formulated as ML problems are classified into the three main categories. For each of the presented methodologies, special emphasis is given to its principles, applications, and limitations.
  arXiv  Detail & Related papers  (2020-08-05T15:45:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.