Deep Iterative 2D/3D Registration

• URL: http://arxiv.org/abs/2107.10004v1
• Date: Wed, 21 Jul 2021 10:51:29 GMT
• Title: Deep Iterative 2D/3D Registration
• Authors: Srikrishna Jaganathan, Jian Wang, Anja Borsdorf, Karthik Shetty, Andreas Maier
• Abstract summary: We propose a novel Deep Learning driven 2D/3D registration framework that can be used end-to-end for iterative registration tasks. We accomplish this by learning the update step of the 2D/3D registration framework using Point-to-Plane Correspondences. Our proposed method achieves an average runtime of around 8s, a mean re-projection distance error of 0.60 $pm$ 0.40 mm with a success ratio of 97 percent and a capture range of 60 mm.
• Score: 9.813316061451392
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Learning-based 2D/3D registration methods are highly robust but often lack the necessary registration accuracy for clinical application. A refinement step using the classical optimization-based 2D/3D registration method applied in combination with Deep Learning-based techniques can provide the required accuracy. However, it also increases the runtime. In this work, we propose a novel Deep Learning driven 2D/3D registration framework that can be used end-to-end for iterative registration tasks without relying on any further refinement step. We accomplish this by learning the update step of the 2D/3D registration framework using Point-to-Plane Correspondences. The update step is learned using iterative residual refinement-based optical flow estimation, in combination with the Point-to-Plane correspondence solver embedded as a known operator. Our proposed method achieves an average runtime of around 8s, a mean re-projection distance error of 0.60 $\pm$ 0.40 mm with a success ratio of 97 percent and a capture range of 60 mm. The combination of high registration accuracy, high robustness, and fast runtime makes our solution ideal for clinical applications.

Related papers

• DELTA: Dense Efficient Long-range 3D Tracking for any video [82.26753323263009]
  We introduce DELTA, a novel method that efficiently tracks every pixel in 3D space, enabling accurate motion estimation across entire videos. Our approach leverages a joint global-local attention mechanism for reduced-resolution tracking, followed by a transformer-based upsampler to achieve high-resolution predictions. Our method provides a robust solution for applications requiring fine-grained, long-term motion tracking in 3D space.
  arXiv  Detail & Related papers  (2024-10-31T17:59:01Z)
• Dual-frame Fluid Motion Estimation with Test-time Optimization and Zero-divergence Loss [9.287932323337163]
  3D particle tracking velocimetry (PTV) is a key technique for analyzing turbulent flow. Deep learning-based methods have achieved impressive accuracy in dual-frame fluid motion estimation. We introduce a new method that is completely self-supervised and notably outperforms its fully-supervised counterparts.
  arXiv  Detail & Related papers  (2024-10-15T18:00:00Z)
• DynaWeightPnP: Toward global real-time 3D-2D solver in PnP without correspondences [7.191124861153032]
  This paper addresses a special Perspective-n-Point (Weight) problem: estimating the optimal pose to align 3D and 2D shapes in real-time without correspondences. Experiments were conducted on a typical case, that is, a 3D-2D centerline registration task within Endovascular Image-Guided Interventions. Results demonstrated that the proposed algorithm achieves registration processing rates of 60 Hz (without post-refinement) and 31 (with post-refinement) with competitive accuracy comparable to existing methods.
  arXiv  Detail & Related papers  (2024-09-27T05:31:33Z)
• 3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking [15.330384668966806]
  State-of-the-art 3D multi-object tracking (MOT) approaches typically rely on non-learned model-based algorithms such as Kalman Filter. We propose 3DMOTFormer, a learned geometry-based 3D MOT framework building upon the transformer architecture. Our approach achieves 71.2% and 68.2% AMOTA on the nuScenes validation and test split, respectively.
  arXiv  Detail & Related papers  (2023-08-12T19:19:58Z)
• UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM [60.575435353047304]
  We present an uncertainty learning framework for dense neural simultaneous localization and mapping (SLAM) We propose an online framework for sensor uncertainty estimation that can be trained in a self-supervised manner from only 2D input data.
  arXiv  Detail & Related papers  (2023-06-19T16:26:25Z)
• Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation [87.54604263202941]
  We propose a tiny deep neural network of which partial layers are iteratively exploited for refining its previous estimations. We employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model. Our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.
  arXiv  Detail & Related papers  (2021-11-11T23:31:34Z)
• 6D Pose Estimation with Combined Deep Learning and 3D Vision Techniques for a Fast and Accurate Object Grasping [0.19686770963118383]
  Real-time robotic grasping is a priority target for highly advanced autonomous systems. This paper proposes a novel method with a 2-stage approach that combines a fast 2D object recognition using a deep neural network. The proposed solution has a potential to perform robustly on real-time applications, requiring both efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-11T15:36:55Z)
• Learning the Update Operator for 2D/3D Image Registration [10.720342813316531]
  preoperative volume can be overlaid over the 2D images using 2D/3D image registration. Deep learning-based 2D/3D registration methods have shown promising results by improving computational efficiency and robustness. We show an improvement of 1.8 times in terms of registration accuracy for the update step prediction compared to learning without the known operator.
  arXiv  Detail & Related papers  (2021-02-04T19:52:59Z)
• Human Body Model Fitting by Learned Gradient Descent [48.79414884222403]
  We propose a novel algorithm for the fitting of 3D human shape to images. We show that this algorithm is fast (avg. 120ms convergence), robust to dataset, and achieves state-of-the-art results on public evaluation datasets.
  arXiv  Detail & Related papers  (2020-08-19T14:26:47Z)
• Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration [66.41922513553367]
  We show that learning-based partial-to-partial registration can be achieved in a one-shot manner. We propose an Optimal Transport layer able to account for occluded points thanks to the use of bins. The resulting OPRNet framework outperforms the state of the art on standard benchmarks.
  arXiv  Detail & Related papers  (2020-06-08T12:35:47Z)
• 3DSSD: Point-based 3D Single Stage Object Detector [61.67928229961813]
  We present a point-based 3D single stage object detector, named 3DSSD, achieving a good balance between accuracy and efficiency. Our method outperforms all state-of-the-art voxel-based single stage methods by a large margin, and has comparable performance to two stage point-based methods as well.
  arXiv  Detail & Related papers  (2020-02-24T12:01:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.