The Impact of Loss Functions and Scene Representations for 3D/2D Registration on Single-view Fluoroscopic X-ray Pose Estimation

• URL: http://arxiv.org/abs/2308.00214v3
• Date: Tue, 27 Feb 2024 10:41:58 GMT
• Title: The Impact of Loss Functions and Scene Representations for 3D/2D Registration on Single-view Fluoroscopic X-ray Pose Estimation
• Authors: Chaochao Zhou, Syed Hasib Akhter Faruqui, Abhinav Patel, Ramez N. Abdalla, Michael C. Hurley, Ali Shaibani, Matthew B. Potts, Babak S. Jahromi, Sameer A. Ansari, Donald R. Cantrell
• Abstract summary: We first develop a differentiable projection rendering framework for the efficient computation of Digitally Reconstructed Radiographs (DRRs) We then perform pose estimation by iterative descent using various candidate loss functions, that quantify the image discrepancy of the synthesized DRR with respect to the ground-truth fluoroscopic X-ray image. Using the Mutual Information loss, a comprehensive evaluation of pose estimation performed on a tomographic X-ray dataset of 50 patients$'$ skulls shows that utilizing either discretized (CBCT) or neural (NeTT/mNeRF) scene representations in DiffProj leads to
• Score: 1.758213853394712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many tasks performed in image-guided procedures can be cast as pose estimation problems, where specific projections are chosen to reach a target in 3D space. In this study, we first develop a differentiable projection (DiffProj) rendering framework for the efficient computation of Digitally Reconstructed Radiographs (DRRs) with automatic differentiability from either Cone-Beam Computerized Tomography (CBCT) or neural scene representations, including two newly proposed methods, Neural Tuned Tomography (NeTT) and masked Neural Radiance Fields (mNeRF). We then perform pose estimation by iterative gradient descent using various candidate loss functions, that quantify the image discrepancy of the synthesized DRR with respect to the ground-truth fluoroscopic X-ray image. Compared to alternative loss functions, the Mutual Information loss function can significantly improve pose estimation accuracy, as it can effectively prevent entrapment in local optima. Using the Mutual Information loss, a comprehensive evaluation of pose estimation performed on a tomographic X-ray dataset of 50 patients$'$ skulls shows that utilizing either discretized (CBCT) or neural (NeTT/mNeRF) scene representations in DiffProj leads to comparable performance in DRR appearance and pose estimation (3D angle errors: mean $\leq$ 3.2{\deg} and 90% quantile $\leq$ 3.4{\deg}), despite the latter often incurring considerable training expenses and time. These findings could be instrumental for selecting appropriate approaches to improve the efficiency and effectiveness of fluoroscopic X-ray pose estimation in widespread image-guided interventions.

Related papers

• Intraoperative 2D/3D Image Registration via Differentiable X-ray Rendering [5.617649111108429]
  We present DiffPose, a self-supervised approach that leverages patient-specific simulation and differentiable physics-based rendering to achieve accurate 2D/3D registration without relying on manually labeled data. DiffPose achieves sub-millimeter accuracy across surgical datasets at intraoperative speeds, improving upon existing unsupervised methods by an order of magnitude and even outperforming supervised baselines.
  arXiv  Detail & Related papers  (2023-12-11T13:05:54Z)
• X-Ray to CT Rigid Registration Using Scene Coordinate Regression [1.1687067206676627]
  This paper proposes a fully automatic registration method that is robust to extreme viewpoints. It is based on a fully convolutional neural network (CNN) that regresses the overlapping coordinates for a given X-ray image. The proposed method achieved an average mean target registration error (mTRE) of 3.79 mm in the 50th percentile of the simulated test dataset and projected mTRE of 9.65 mm in the 50th percentile of real fluoroscopic images for pelvis registration.
  arXiv  Detail & Related papers  (2023-11-25T17:48:46Z)
• Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization [56.95046107046027]
  We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for scene coordinate regression. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain.
  arXiv  Detail & Related papers  (2023-10-10T20:11:13Z)
• Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images [59.85702949046042]
  We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process. We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications.
  arXiv  Detail & Related papers  (2023-08-03T21:56:50Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)
• Acute ischemic stroke lesion segmentation in non-contrast CT images using 3D convolutional neural networks [0.0]
  We propose an automatic algorithm aimed at volumetric segmentation of acute ischemic stroke lesion in non-contrast computed tomography brain 3D images. Our deep-learning approach is based on the popular 3D U-Net convolutional neural network architecture.
  arXiv  Detail & Related papers  (2023-01-17T10:39:39Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Probabilistic 3D surface reconstruction from sparse MRI information [58.14653650521129]
  We present a novel probabilistic deep learning approach for concurrent 3D surface reconstruction from sparse 2D MR image data and aleatoric uncertainty prediction. Our method is capable of reconstructing large surface meshes from three quasi-orthogonal MR imaging slices from limited training sets.
  arXiv  Detail & Related papers  (2020-10-05T14:18:52Z)
• Simultaneous Estimation of X-ray Back-Scatter and Forward-Scatter using Multi-Task Learning [59.17383024536595]
  Back-scatter significantly contributes to patient (skin) dose during complicated interventions. Forward-scattered radiation reduces contrast in projection images and introduces artifacts in 3-D reconstructions. We propose a novel approach combining conventional techniques with learning-based methods to simultaneously estimate the forward-scatter reaching the detector.
  arXiv  Detail & Related papers  (2020-07-08T10:47:37Z)
• 3D Probabilistic Segmentation and Volumetry from 2D projection images [10.32519161805588]
  X-Ray imaging is quick, cheap and useful for front-line care assessment and intra-operative real-time imaging. It suffers from projective information loss and lacks vital information on which many essential diagnostic biomarkers are based. In this paper we explore probabilistic methods to reconstruct 3D volumetric images from 2D imaging modalities.
  arXiv  Detail & Related papers  (2020-06-23T08:00:51Z)
• Robust Self-Supervised Learning of Deterministic Errors in Single-Plane (Monoplanar) and Dual-Plane (Biplanar) X-ray Fluoroscopy [2.7528170226206443]
  Fluoroscopic imaging that captures X-ray images at video framerates is advantageous for guiding catheter insertions by vascular surgeons and interventional radiologists. Visualizing the dynamical movements non-invasively allows complex surgical procedures to be performed with less trauma to the patient. This paper presents a robust self-calibration algorithm suitable for single-plane and dual-plane fluoroscopy.
  arXiv  Detail & Related papers  (2020-01-03T01:56:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.