X-Ray to CT Rigid Registration Using Scene Coordinate Regression
- URL: http://arxiv.org/abs/2311.15087v1
- Date: Sat, 25 Nov 2023 17:48:46 GMT
- Title: X-Ray to CT Rigid Registration Using Scene Coordinate Regression
- Authors: Pragyan Shrestha, Chun Xie, Hidehiko Shishido, Yuichi Yoshii, Itary
Kitahara
- Abstract summary: This paper proposes a fully automatic registration method that is robust to extreme viewpoints.
It is based on a fully convolutional neural network (CNN) that regresses the overlapping coordinates for a given X-ray image.
The proposed method achieved an average mean target registration error (mTRE) of 3.79 mm in the 50th percentile of the simulated test dataset and projected mTRE of 9.65 mm in the 50th percentile of real fluoroscopic images for pelvis registration.
- Score: 1.1687067206676627
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Intraoperative fluoroscopy is a frequently used modality in minimally
invasive orthopedic surgeries. Aligning the intraoperatively acquired X-ray
image with the preoperatively acquired 3D model of a computed tomography (CT)
scan reduces the mental burden on surgeons induced by the overlapping
anatomical structures in the acquired images. This paper proposes a fully
automatic registration method that is robust to extreme viewpoints and does not
require manual annotation of landmark points during training. It is based on a
fully convolutional neural network (CNN) that regresses the scene coordinates
for a given X-ray image. The scene coordinates are defined as the intersection
of the back-projected rays from a pixel toward the 3D model. Training data for
a patient-specific model were generated through a realistic simulation of a
C-arm device using preoperative CT scans. In contrast, intraoperative
registration was achieved by solving the perspective-n-point (PnP) problem with
a random sample and consensus (RANSAC) algorithm. Experiments were conducted
using a pelvic CT dataset that included several real fluoroscopic (X-ray)
images with ground truth annotations. The proposed method achieved an average
mean target registration error (mTRE) of 3.79 mm in the 50th percentile of the
simulated test dataset and projected mTRE of 9.65 mm in the 50th percentile of
real fluoroscopic images for pelvis registration.
Related papers
- SurgPointTransformer: Vertebrae Shape Completion with RGB-D Data [0.0]
This study introduces an alternative, radiation-free approach for reconstructing the 3D spine anatomy using RGB-D data.
We introduce SurgPointTransformer, a shape completion approach for surgical applications that can accurately reconstruct the unexposed spine regions from sparse observations of the exposed surface.
Our method significantly outperforms the state-of-the-art baselines, achieving an average Chamfer Distance of 5.39, an F-Score of 0.85, an Earth Mover's Distance of 0.011, and a Signal-to-Noise Ratio of 22.90 dB.
arXiv Detail & Related papers (2024-10-02T11:53:28Z) - Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering.
Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively.
We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
arXiv Detail & Related papers (2024-09-18T13:40:59Z) - Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data [9.21828361691977]
This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries.
It shows an approach for generating 3D anatomical models of the spine from only a few fluoroscopic images.
It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research.
arXiv Detail & Related papers (2024-01-29T10:22:45Z) - Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering [5.617649111108429]
We present DiffPose, a self-supervised approach that leverages patient-specific simulation and differentiable physics-based rendering to achieve accurate 2D/3D registration without relying on manually labeled data.
DiffPose achieves sub-millimeter accuracy across surgical datasets at intraoperative speeds, improving upon existing unsupervised methods by an order of magnitude and even outperforming supervised baselines.
arXiv Detail & Related papers (2023-12-11T13:05:54Z) - AiAReSeg: Catheter Detection and Segmentation in Interventional
Ultrasound using Transformers [75.20925220246689]
endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature.
This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
arXiv Detail & Related papers (2023-09-25T19:34:12Z) - On the Localization of Ultrasound Image Slices within Point Distribution
Models [84.27083443424408]
Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US)
Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology.
We present a framework for automated US image slice localization within a 3D shape representation.
arXiv Detail & Related papers (2023-09-01T10:10:46Z) - Simulation-Driven Training of Vision Transformers Enabling Metal
Segmentation in X-Ray Images [6.416928579907334]
This study proposes to generate simulated X-ray images based on CT data sets combined with computer aided design (CAD) implants.
The metal segmentation in CBCT projections serves as a prerequisite for metal artifact avoidance and reduction algorithms.
Our study indicates that the CAD model-based data generation has high flexibility and could be a way to overcome the problem of shortage in clinical data sampling and labelling.
arXiv Detail & Related papers (2022-03-17T09:58:58Z) - XraySyn: Realistic View Synthesis From a Single Radiograph Through CT
Priors [118.27130593216096]
A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane.
To the best of our knowledge, this is the first work on radiograph view synthesis.
We show that by gaining an understanding of radiography in 3D space, our method can be applied to radiograph bone extraction and suppression without groundtruth bone labels.
arXiv Detail & Related papers (2020-12-04T05:08:53Z) - Pose-dependent weights and Domain Randomization for fully automatic
X-ray to CT Registration [51.280096834264256]
Fully automatic X-ray to CT registration requires an initial alignment within the capture range of existing intensity-based registrations.
This work provides a novel automatic initialization, which enables end to end registration.
The mean (+-standard deviation) target registration error in millimetres is 4.1 +- 4.3 for simulated X-rays with a success rate of 92% and 4.2 +- 3.9 for real X-rays with a success rate of 86.8%, where a success is defined as a translation error of less than 30mm.
arXiv Detail & Related papers (2020-11-14T12:50:32Z) - Tattoo tomography: Freehand 3D photoacoustic image reconstruction with
an optical pattern [49.240017254888336]
Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties.
A current drawback is the limited field-of-view provided by the conventionally applied 2D probes.
We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
arXiv Detail & Related papers (2020-11-10T09:27:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.