An objective comparison of methods for augmented reality in laparoscopic
liver resection by preoperative-to-intraoperative image fusion
- URL: http://arxiv.org/abs/2401.15753v2
- Date: Wed, 7 Feb 2024 11:47:38 GMT
- Title: An objective comparison of methods for augmented reality in laparoscopic
liver resection by preoperative-to-intraoperative image fusion
- Authors: Sharib Ali, Yamid Espinel, Yueming Jin, Peng Liu, Bianca G\"uttner,
Xukun Zhang, Lihua Zhang, Tom Dowrick, Matthew J. Clarkson, Shiting Xiao,
Yifan Wu, Yijun Yang, Lei Zhu, Dai Sun, Lan Li, Micha Pfeiffer, Shahid Farid,
Lena Maier-Hein, Emmanuel Buc, Adrien Bartoli
- Abstract summary: Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image.
Most of the algorithms make use of anatomical landmarks to guide registration.
These landmarks include the liver's inferior ridge, the falciform ligament, and the occluding contours.
We present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge (P2ILF), which investigates the possibilities of detecting these landmarks automatically and using them in registration.
- Score: 33.12510773034339
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Augmented reality for laparoscopic liver resection is a visualisation mode
that allows a surgeon to localise tumours and vessels embedded within the liver
by projecting them on top of a laparoscopic image. Preoperative 3D models
extracted from CT or MRI data are registered to the intraoperative laparoscopic
images during this process. In terms of 3D-2D fusion, most of the algorithms
make use of anatomical landmarks to guide registration. These landmarks include
the liver's inferior ridge, the falciform ligament, and the occluding contours.
They are usually marked by hand in both the laparoscopic image and the 3D
model, which is time-consuming and may contain errors if done by a
non-experienced user. Therefore, there is a need to automate this process so
that augmented reality can be used effectively in the operating room. We
present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge
(P2ILF), held during the Medical Imaging and Computer Assisted Interventions
(MICCAI 2022) conference, which investigates the possibilities of detecting
these landmarks automatically and using them in registration. The challenge was
divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D
registration task. The teams were provided with training data consisting of 167
laparoscopic images and 9 preoperative 3D models from 9 patients, with the
corresponding 2D and 3D landmark annotations. A total of 6 teams from 4
countries participated, whose proposed methods were evaluated on 16 images and
two preoperative 3D models from two patients. All the teams proposed deep
learning-based methods for the 2D and 3D landmark segmentation tasks and
differentiable rendering-based methods for the registration task. Based on the
experimental outcomes, we propose three key hypotheses that determine current
limitations and future directions for research in this domain.
Related papers
- SurgPointTransformer: Vertebrae Shape Completion with RGB-D Data [0.0]
This study introduces an alternative, radiation-free approach for reconstructing the 3D spine anatomy using RGB-D data.
We introduce SurgPointTransformer, a shape completion approach for surgical applications that can accurately reconstruct the unexposed spine regions from sparse observations of the exposed surface.
Our method significantly outperforms the state-of-the-art baselines, achieving an average Chamfer Distance of 5.39, an F-Score of 0.85, an Earth Mover's Distance of 0.011, and a Signal-to-Noise Ratio of 22.90 dB.
arXiv Detail & Related papers (2024-10-02T11:53:28Z) - Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering.
Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively.
We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
arXiv Detail & Related papers (2024-09-18T13:40:59Z) - Depth-Driven Geometric Prompt Learning for Laparoscopic Liver Landmark Detection [43.600236988802465]
Liver anatomical landmarks serve as important markers for 2D-3D alignment.
To facilitate the detection of laparoscopic liver landmarks, we collect a novel dataset called L3D.
We propose a depth-driven geometric prompt learning network, namely D2GPLand.
arXiv Detail & Related papers (2024-06-25T18:02:11Z) - Creating a Digital Twin of Spinal Surgery: A Proof of Concept [68.37190859183663]
Surgery digitalization is the process of creating a virtual replica of real-world surgery.
We present a proof of concept (PoC) for surgery digitalization that is applied to an ex-vivo spinal surgery.
We employ five RGB-D cameras for dynamic 3D reconstruction of the surgeon, a high-end camera for 3D reconstruction of the anatomy, an infrared stereo camera for surgical instrument tracking, and a laser scanner for 3D reconstruction of the operating room and data fusion.
arXiv Detail & Related papers (2024-03-25T13:09:40Z) - Generative Enhancement for 3D Medical Images [74.17066529847546]
We propose GEM-3D, a novel generative approach to the synthesis of 3D medical images.
Our method begins with a 2D slice, noted as the informed slice to serve the patient prior, and propagates the generation process using a 3D segmentation mask.
By decomposing the 3D medical images into masks and patient prior information, GEM-3D offers a flexible yet effective solution for generating versatile 3D images.
arXiv Detail & Related papers (2024-03-19T15:57:04Z) - Monocular Microscope to CT Registration using Pose Estimation of the
Incus for Augmented Reality Cochlear Implant Surgery [3.8909273404657556]
We develop a method that permits direct 2D-to-3D registration of the view microscope video to the pre-operative Computed Tomography (CT) scan without the need for external tracking equipment.
Our results demonstrate the accuracy with an average rotation error of less than 25 degrees and a translation error of less than 2 mm, 3 mm, and 0.55% for the x, y, and z axes, respectively.
arXiv Detail & Related papers (2024-03-12T00:26:08Z) - On the Localization of Ultrasound Image Slices within Point Distribution
Models [84.27083443424408]
Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US)
Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology.
We present a framework for automated US image slice localization within a 3D shape representation.
arXiv Detail & Related papers (2023-09-01T10:10:46Z) - Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose
Estimation of Surgical Instruments [66.74633676595889]
We present a multi-camera capture setup consisting of static and head-mounted cameras.
Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.
Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
arXiv Detail & Related papers (2023-05-05T13:42:19Z) - CNN-based real-time 2D-3D deformable registration from a single X-ray
projection [2.1198879079315573]
This paper presents a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image.
A dataset composed of displacement fields and 2D projections of the anatomy is generated from a preoperative scan.
A neural network is trained to recover the unknown 3D displacement field from a single projection image.
arXiv Detail & Related papers (2022-12-15T09:57:19Z) - View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection.
First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan.
Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view.
Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
arXiv Detail & Related papers (2022-09-20T11:58:23Z) - IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable
Registration [1.2246649738388387]
We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image.
We show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from radiographs with clinically acceptable accuracy.
arXiv Detail & Related papers (2021-10-31T12:48:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.