An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models
- URL: http://arxiv.org/abs/2402.11840v1
- Date: Mon, 19 Feb 2024 05:06:52 GMT
- Title: An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models
- Authors: Jan Emily Mangulabnan, Roger D. Soberanis-Mukul, Timo Teufel, Manish
Sahu, Jose L. Porras, S. Swaroop Vedula, Masaru Ishii, Gregory Hager, Russell
H. Taylor, Mathias Unberath
- Abstract summary: We propose a first vision-based approach to update the preoperative 3D anatomical model.
Results show a decrease in error during surgical progression as opposed to increasing when no update is employed.
- Score: 8.516340459721484
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Purpose: Preoperative imaging plays a pivotal role in sinus surgery where CTs
offer patient-specific insights of complex anatomy, enabling real-time
intraoperative navigation to complement endoscopy imaging. However, surgery
elicits anatomical changes not represented in the preoperative model,
generating an inaccurate basis for navigation during surgery progression.
Methods: We propose a first vision-based approach to update the preoperative
3D anatomical model leveraging intraoperative endoscopic video for navigated
sinus surgery where relative camera poses are known. We rely on comparisons of
intraoperative monocular depth estimates and preoperative depth renders to
identify modified regions. The new depths are integrated in these regions
through volumetric fusion in a truncated signed distance function
representation to generate an intraoperative 3D model that reflects tissue
manipulation.
Results: We quantitatively evaluate our approach by sequentially updating
models for a five-step surgical progression in an ex vivo specimen. We compute
the error between correspondences from the updated model and ground-truth
intraoperative CT in the region of anatomical modification. The resulting
models show a decrease in error during surgical progression as opposed to
increasing when no update is employed.
Conclusion: Our findings suggest that preoperative 3D anatomical models can
be updated using intraoperative endoscopy video in navigated sinus surgery.
Future work will investigate improvements to monocular depth estimation as well
as removing the need for external navigation systems. The resulting ability to
continuously update the patient model may provide surgeons with a more precise
understanding of the current anatomical state and paves the way toward a
digital twin paradigm for sinus surgery.
Related papers
- High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map.
Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface.
This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
arXiv Detail & Related papers (2024-04-20T18:06:26Z) - Creating a Digital Twin of Spinal Surgery: A Proof of Concept [68.37190859183663]
Surgery digitalization is the process of creating a virtual replica of real-world surgery.
We present a proof of concept (PoC) for surgery digitalization that is applied to an ex-vivo spinal surgery.
We employ five RGB-D cameras for dynamic 3D reconstruction of the surgeon, a high-end camera for 3D reconstruction of the anatomy, an infrared stereo camera for surgical instrument tracking, and a laser scanner for 3D reconstruction of the operating room and data fusion.
arXiv Detail & Related papers (2024-03-25T13:09:40Z) - EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting [53.38166294158047]
EndoGSLAM is an efficient approach for endoscopic surgeries, which integrates streamlined representation and differentiable Gaussianization.
Experiments show that EndoGSLAM achieves a better trade-off between intraoperative availability and reconstruction quality than traditional or neural SLAM approaches.
arXiv Detail & Related papers (2024-03-22T11:27:43Z) - Monocular Microscope to CT Registration using Pose Estimation of the
Incus for Augmented Reality Cochlear Implant Surgery [3.8909273404657556]
We develop a method that permits direct 2D-to-3D registration of the view microscope video to the pre-operative Computed Tomography (CT) scan without the need for external tracking equipment.
Our results demonstrate the accuracy with an average rotation error of less than 25 degrees and a translation error of less than 2 mm, 3 mm, and 0.55% for the x, y, and z axes, respectively.
arXiv Detail & Related papers (2024-03-12T00:26:08Z) - An objective comparison of methods for augmented reality in laparoscopic
liver resection by preoperative-to-intraoperative image fusion [33.12510773034339]
Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image.
Most of the algorithms make use of anatomical landmarks to guide registration.
These landmarks include the liver's inferior ridge, the falciform ligament, and the occluding contours.
We present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge (P2ILF), which investigates the possibilities of detecting these landmarks automatically and using them in registration.
arXiv Detail & Related papers (2024-01-28T20:30:14Z) - Surgical-DINO: Adapter Learning of Foundation Models for Depth
Estimation in Endoscopic Surgery [12.92291406687467]
We design a foundation model-based depth estimation method, referred to as Surgical-DINO, a low-rank adaptation of the DINOv2 for depth estimation in endoscopic surgery.
We build LoRA layers and integrate them into DINO to adapt with surgery-specific domain knowledge instead of conventional fine-tuning.
Our model is extensively validated on a MICCAI challenge dataset of SCARED, which is collected from da Vinci Xi endoscope surgery.
arXiv Detail & Related papers (2024-01-11T16:22:42Z) - A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy
from Monocular Endoscopic Video [8.32570164101507]
We perform a quantitative analysis of a self-supervised approach for sinus reconstruction using endoscopic sequences and optical tracking.
Our results show that the generated reconstructions are in high agreement with the anatomy, yielding an average point-to-mesh error of 0.91 mm.
We identify that pose and depth estimation inaccuracies contribute equally to this error and that locally consistent sequences with shorter trajectories generate more accurate reconstructions.
arXiv Detail & Related papers (2023-10-22T17:11:40Z) - AiAReSeg: Catheter Detection and Segmentation in Interventional
Ultrasound using Transformers [75.20925220246689]
endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature.
This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
arXiv Detail & Related papers (2023-09-25T19:34:12Z) - Live image-based neurosurgical guidance and roadmap generation using
unsupervised embedding [53.992124594124896]
We present a method for live image-only guidance leveraging a large data set of annotated neurosurgical videos.
A generated roadmap encodes the common anatomical paths taken in surgeries in the training set.
We trained and evaluated the proposed method with a data set of 166 transsphenoidal adenomectomy procedures.
arXiv Detail & Related papers (2023-03-31T12:52:24Z) - Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical
Procedures [70.69948035469467]
We take advantage of the latest computer vision methodologies for generating 3D graphs from camera views.
We then introduce the Multimodal Semantic Graph Scene (MSSG) which aims at providing unified symbolic and semantic representation of surgical procedures.
arXiv Detail & Related papers (2021-06-09T14:35:44Z) - Tattoo tomography: Freehand 3D photoacoustic image reconstruction with
an optical pattern [49.240017254888336]
Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties.
A current drawback is the limited field-of-view provided by the conventionally applied 2D probes.
We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
arXiv Detail & Related papers (2020-11-10T09:27:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.