Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints

• URL: http://arxiv.org/abs/2506.22191v1
• Date: Fri, 27 Jun 2025 12:57:58 GMT
• Title: Robust and Accurate Multi-view 2D/3D Image Registration with Differentiable X-ray Rendering and Dual Cross-view Constraints
• Authors: Yuxin Cui, Rui Song, Yibin Li, Max Q. -H. Meng, Zhe Min,
• Abstract summary: We propose a novel multi-view 2D/3D rigid registration approach comprising two stages.<n>In the first stage, a combined loss function is designed, incorporating both the differences between predicted and ground-truth poses.<n>In the second stage, test-time optimization is performed to refine the estimated poses from the coarse stage.
• Score: 45.57808049168089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust and accurate 2D/3D registration, which aligns preoperative models with intraoperative images of the same anatomy, is crucial for successful interventional navigation. To mitigate the challenge of a limited field of view in single-image intraoperative scenarios, multi-view 2D/3D registration is required by leveraging multiple intraoperative images. In this paper, we propose a novel multi-view 2D/3D rigid registration approach comprising two stages. In the first stage, a combined loss function is designed, incorporating both the differences between predicted and ground-truth poses and the dissimilarities (e.g., normalized cross-correlation) between simulated and observed intraoperative images. More importantly, additional cross-view training loss terms are introduced for both pose and image losses to explicitly enforce cross-view constraints. In the second stage, test-time optimization is performed to refine the estimated poses from the coarse stage. Our method exploits the mutual constraints of multi-view projection poses to enhance the robustness of the registration process. The proposed framework achieves a mean target registration error (mTRE) of $0.79 \pm 2.17$ mm on six specimens from the DeepFluoro dataset, demonstrating superior performance compared to state-of-the-art registration algorithms.

Related papers

• Learning to Align and Refine: A Foundation-to-Diffusion Framework for Occlusion-Robust Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a dual-stage Foundation-to-Diffusion framework that precisely align 2D prior guidance from vision foundation models.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• Diff-Reg v2: Diffusion-Based Matching Matrix Estimation for Image Matching and 3D Registration [33.8118117906136]
  We introduce an innovative paradigm that leverages a diffusion model in matrix space for robust matching matrix estimation.<n>Specifically, we apply the diffusion model in the doubly matrix space for 3D-3D and 2D-3D registration tasks.<n>For all three registration tasks, we provide adaptive matching matrix embedding implementations tailored to the specific characteristics of each task.
  arXiv  Detail & Related papers  (2025-03-06T06:13:27Z)
• Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization [22.633633605566214]
  We propose an unsupervised multimodal medical image registration method based on multilevel correlation balanced optimization.<n>For preoperative medical images in different modalities, the alignment and stacking of valid information is achieved by the maximum fusion between deformation fields.
  arXiv  Detail & Related papers  (2024-09-08T09:38:59Z)
• Breaking Modality Disparity: Harmonized Representation for Infrared and Visible Image Registration [66.33746403815283]
  We propose a scene-adaptive infrared and visible image registration. We employ homography to simulate the deformation between different planes. We propose the first ground truth available misaligned infrared and visible image dataset.
  arXiv  Detail & Related papers  (2023-04-12T06:49:56Z)
• Progressive Multi-view Human Mesh Recovery with Self-Supervision [68.60019434498703]
  Existing solutions typically suffer from poor generalization performance to new settings. We propose a novel simulation-based training pipeline for multi-view human mesh recovery.
  arXiv  Detail & Related papers  (2022-12-10T06:28:29Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Multi-Objective Dual Simplex-Mesh Based Deformable Image Registration for 3D Medical Images -- Proof of Concept [0.7734726150561088]
  This work introduces the first method for multi-objective 3D deformable image registration, using a 3D dual-dynamic grid transformation model based on simplex meshes. Our proof-of-concept prototype shows promising results on synthetic and clinical 3D registration problems.
  arXiv  Detail & Related papers  (2022-02-22T16:07:29Z)
• The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective [6.669432838047949]
  Image-based navigation is widely considered the next frontier of minimally invasive surgery. 2D/3D registration is a technique to estimate the spatial relationships between 3D structures and 2D images. Recent advent of machine learning-based approaches to imaging problems holds promise for solving some of the notorious challenges in 2D/3D registration.
  arXiv  Detail & Related papers  (2021-08-04T18:31:29Z)
• End-to-end Ultrasound Frame to Volume Registration [9.738024231762465]
  We propose an end-to-end frame-to-volume registration network (FVR-Net) for 2D and 3D registration. Our model shows superior efficiency for real-time interventional guidance with highly competitive registration accuracy.
  arXiv  Detail & Related papers  (2021-07-14T01:59:42Z)
• Recurrent Multi-view Alignment Network for Unsupervised Surface Registration [79.72086524370819]
  Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data. We propose to represent the non-rigid transformation with a point-wise combination of several rigid transformations. We also introduce a differentiable loss function that measures the 3D shape similarity on the projected multi-view 2D depth images.
  arXiv  Detail & Related papers  (2020-11-24T14:22:42Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.