Deep Iterative 2D/3D Registration
- URL: http://arxiv.org/abs/2107.10004v1
- Date: Wed, 21 Jul 2021 10:51:29 GMT
- Title: Deep Iterative 2D/3D Registration
- Authors: Srikrishna Jaganathan, Jian Wang, Anja Borsdorf, Karthik Shetty,
Andreas Maier
- Abstract summary: We propose a novel Deep Learning driven 2D/3D registration framework that can be used end-to-end for iterative registration tasks.
We accomplish this by learning the update step of the 2D/3D registration framework using Point-to-Plane Correspondences.
Our proposed method achieves an average runtime of around 8s, a mean re-projection distance error of 0.60 $pm$ 0.40 mm with a success ratio of 97 percent and a capture range of 60 mm.
- Score: 9.813316061451392
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Learning-based 2D/3D registration methods are highly robust but often
lack the necessary registration accuracy for clinical application. A refinement
step using the classical optimization-based 2D/3D registration method applied
in combination with Deep Learning-based techniques can provide the required
accuracy. However, it also increases the runtime. In this work, we propose a
novel Deep Learning driven 2D/3D registration framework that can be used
end-to-end for iterative registration tasks without relying on any further
refinement step. We accomplish this by learning the update step of the 2D/3D
registration framework using Point-to-Plane Correspondences. The update step is
learned using iterative residual refinement-based optical flow estimation, in
combination with the Point-to-Plane correspondence solver embedded as a known
operator. Our proposed method achieves an average runtime of around 8s, a mean
re-projection distance error of 0.60 $\pm$ 0.40 mm with a success ratio of 97
percent and a capture range of 60 mm. The combination of high registration
accuracy, high robustness, and fast runtime makes our solution ideal for
clinical applications.
Related papers
- 3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking [15.330384668966806]
State-of-the-art 3D multi-object tracking (MOT) approaches typically rely on non-learned model-based algorithms such as Kalman Filter.
We propose 3DMOTFormer, a learned geometry-based 3D MOT framework building upon the transformer architecture.
Our approach achieves 71.2% and 68.2% AMOTA on the nuScenes validation and test split, respectively.
arXiv Detail & Related papers (2023-08-12T19:19:58Z) - UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM [60.575435353047304]
We present an uncertainty learning framework for dense neural simultaneous localization and mapping (SLAM)
We propose an online framework for sensor uncertainty estimation that can be trained in a self-supervised manner from only 2D input data.
arXiv Detail & Related papers (2023-06-19T16:26:25Z) - SENDD: Sparse Efficient Neural Depth and Deformation for Tissue Tracking [7.282909831316735]
Real-time estimation of 3D tissue motion is essential for robotically assisted surgery.
Our model, Sparse Efficient Neural Depth and Deformation (SENDD), extends prior 2D tracking work to estimate flow in 3D space.
SENDD does this by using graph neural networks of sparse keypoint matches to estimate both depth and 3D flow anywhere.
arXiv Detail & Related papers (2023-05-10T22:04:56Z) - GOOD: General Optimization-based Fusion for 3D Object Detection via
LiDAR-Camera Object Candidates [10.534984939225014]
3D object detection serves as the core basis of the perception tasks in autonomous driving.
Good is a general optimization-based fusion framework that can achieve satisfying detection without training additional models.
Experiments on both nuScenes and KITTI datasets are carried out and the results show that GOOD outperforms by 9.1% on mAP score compared with PointPillars.
arXiv Detail & Related papers (2023-03-17T07:05:04Z) - Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation [87.54604263202941]
We propose a tiny deep neural network of which partial layers are iteratively exploited for refining its previous estimations.
We employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model.
Our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.
arXiv Detail & Related papers (2021-11-11T23:31:34Z) - 6D Pose Estimation with Combined Deep Learning and 3D Vision Techniques
for a Fast and Accurate Object Grasping [0.19686770963118383]
Real-time robotic grasping is a priority target for highly advanced autonomous systems.
This paper proposes a novel method with a 2-stage approach that combines a fast 2D object recognition using a deep neural network.
The proposed solution has a potential to perform robustly on real-time applications, requiring both efficiency and accuracy.
arXiv Detail & Related papers (2021-11-11T15:36:55Z) - Soft Expectation and Deep Maximization for Image Feature Detection [68.8204255655161]
We propose SEDM, an iterative semi-supervised learning process that flips the question and first looks for repeatable 3D points, then trains a detector to localize them in image space.
Our results show that this new model trained using SEDM is able to better localize the underlying 3D points in a scene.
arXiv Detail & Related papers (2021-04-21T00:35:32Z) - Learning the Update Operator for 2D/3D Image Registration [10.720342813316531]
preoperative volume can be overlaid over the 2D images using 2D/3D image registration.
Deep learning-based 2D/3D registration methods have shown promising results by improving computational efficiency and robustness.
We show an improvement of 1.8 times in terms of registration accuracy for the update step prediction compared to learning without the known operator.
arXiv Detail & Related papers (2021-02-04T19:52:59Z) - Human Body Model Fitting by Learned Gradient Descent [48.79414884222403]
We propose a novel algorithm for the fitting of 3D human shape to images.
We show that this algorithm is fast (avg. 120ms convergence), robust to dataset, and achieves state-of-the-art results on public evaluation datasets.
arXiv Detail & Related papers (2020-08-19T14:26:47Z) - Learning 3D-3D Correspondences for One-shot Partial-to-partial
Registration [66.41922513553367]
We show that learning-based partial-to-partial registration can be achieved in a one-shot manner.
We propose an Optimal Transport layer able to account for occluded points thanks to the use of bins.
The resulting OPRNet framework outperforms the state of the art on standard benchmarks.
arXiv Detail & Related papers (2020-06-08T12:35:47Z) - 3DSSD: Point-based 3D Single Stage Object Detector [61.67928229961813]
We present a point-based 3D single stage object detector, named 3DSSD, achieving a good balance between accuracy and efficiency.
Our method outperforms all state-of-the-art voxel-based single stage methods by a large margin, and has comparable performance to two stage point-based methods as well.
arXiv Detail & Related papers (2020-02-24T12:01:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.