Semantic 3D Reconstructions with SLAM for Central Airway Obstruction

• URL: http://arxiv.org/abs/2509.13541v1
• Date: Tue, 16 Sep 2025 21:14:16 GMT
• Title: Semantic 3D Reconstructions with SLAM for Central Airway Obstruction
• Authors: Ayberk Acar, Fangjie Li, Hao Li, Lidia Al-Zogbi, Kanyifeechukwu Jane Oguine, Susheela Sharma Stern, Jesse F. d'Almeida, Robert J. Webster III, Ipek Oguz, Jie Ying Wu,
• Abstract summary: Central airway obstruction (CAO) is a life-threatening condition with increasing incidence.<n>Traditional treatment methods such as bronchoscopy and electrocautery can be used to remove the tumor completely.<n>Recent advances allow robotic interventions with lesser risk.<n>We present a novel pipeline that enables real-time, semantically informed 3D reconstructions of the central airway using monocular endoscopic video.
• Score: 4.925599926643841
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Central airway obstruction (CAO) is a life-threatening condition with increasing incidence, caused by tumors in and outside of the airway. Traditional treatment methods such as bronchoscopy and electrocautery can be used to remove the tumor completely; however, these methods carry a high risk of complications. Recent advances allow robotic interventions with lesser risk. The combination of robot interventions with scene understanding and mapping also opens up the possibilities for automation. We present a novel pipeline that enables real-time, semantically informed 3D reconstructions of the central airway using monocular endoscopic video. Our approach combines DROID-SLAM with a segmentation model trained to identify obstructive tissues. The SLAM module reconstructs the 3D geometry of the airway in real time, while the segmentation masks guide the annotation of obstruction regions within the reconstructed point cloud. To validate our pipeline, we evaluate the reconstruction quality using ex vivo models. Qualitative and quantitative results show high similarity between ground truth CT scans and the 3D reconstructions (0.62 mm Chamfer distance). By integrating segmentation directly into the SLAM workflow, our system produces annotated 3D maps that highlight clinically relevant regions in real time. High-speed capabilities of the pipeline allows quicker reconstructions compared to previous work, reflecting the surgical scene more accurately. To the best of our knowledge, this is the first work to integrate semantic segmentation with real-time monocular SLAM for endoscopic CAO scenarios. Our framework is modular and can generalize to other anatomies or procedures with minimal changes, offering a promising step toward autonomous robotic interventions.

Related papers

• Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection [50.388465935739376]
  Liver registration by overlaying preoperative 3D models onto intraoperative 2D frames can assist surgeons in perceiving the spatial anatomy of the liver clearly for a higher surgical success rate.<n>Existing registration methods rely heavily on anatomical landmark-based, which encounter two major limitations.<n>We propose a landmark-free preoperative-to-intraoperative registration framework utilizing effective self-supervised learning.
  arXiv  Detail & Related papers  (2025-04-21T14:55:57Z)
• Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video [35.241054116681426]
  Endo3R is a unified 3D foundation model for online scale-consistent reconstruction from monocular surgical video.<n>Our model unifies the tasks by predicting globally aligned pointmaps, scale-consistent video depths, and camera parameters without any offline optimization.
  arXiv  Detail & Related papers  (2025-04-04T06:05:22Z)
• From Monocular Vision to Autonomous Action: Guiding Tumor Resection via 3D Reconstruction [8.485401463053098]
  We propose a 3D mapping pipeline that uses only RGB images to create segmented point clouds of the target anatomy.<n>In several metrics, including post-procedure tissue model evaluation, our pipeline performs comparably to RGB-D cameras.<n>This study is a step toward the complete autonomy of surgical robots.
  arXiv  Detail & Related papers  (2025-03-20T15:57:18Z)
• MedTet: An Online Motion Model for 4D Heart Reconstruction [59.74234226055964]
  We present a novel approach to reconstruction of 3D cardiac motion from sparse intraoperative data.<n>Existing methods can accurately reconstruct 3D organ geometries from full 3D volumetric imaging.<n>We propose a versatile framework for reconstructing 3D motion from such partial data.
  arXiv  Detail & Related papers  (2024-12-03T17:18:33Z)
• EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting [53.38166294158047]
  EndoGSLAM is an efficient approach for endoscopic surgeries, which integrates streamlined representation and differentiable Gaussianization. Experiments show that EndoGSLAM achieves a better trade-off between intraoperative availability and reconstruction quality than traditional or neural SLAM approaches.
  arXiv  Detail & Related papers  (2024-03-22T11:27:43Z)
• An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models [8.516340459721484]
  We propose a first vision-based approach to update the preoperative 3D anatomical model. Results show a decrease in error during surgical progression as opposed to increasing when no update is employed.
  arXiv  Detail & Related papers  (2024-02-19T05:06:52Z)
• RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection [4.992840879935547]
  RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking.
  arXiv  Detail & Related papers  (2023-10-12T07:10:12Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images [39.94162291765236]
  We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process.
  arXiv  Detail & Related papers  (2023-08-03T21:56:50Z)
• A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning [56.52933974838905]
  Current medical workflow requires manual delineation of organs-at-risk (OAR) In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation. Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
  arXiv  Detail & Related papers  (2022-03-01T17:08:41Z)
• Deep Negative Volume Segmentation [60.44793799306154]
  We propose a new angle to the 3D segmentation task: segment empty spaces between all the tissues surrounding the object. Our approach is an end-to-end pipeline that comprises a V-Net for bone segmentation. We validate the idea on the CT scans in a 50-patient dataset, annotated by experts in maxillofacial medicine.
  arXiv  Detail & Related papers  (2020-06-22T16:55:23Z)
• Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment [23.54387940230938]
  Reconstructing accurate 3D surface models of sinus anatomy directly from an endoscopic video is a promising avenue for cross-sectional and longitudinal analysis. We present a patient-specific, learning-based method for 3D reconstruction of sinus surface anatomy directly and only from endoscopic videos.
  arXiv  Detail & Related papers  (2020-03-18T23:11:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.