Towards real-time 6D pose estimation of objects in single-view cone-beam X-ray

• URL: http://arxiv.org/abs/2211.03211v1
• Date: Sun, 6 Nov 2022 20:06:28 GMT
• Title: Towards real-time 6D pose estimation of objects in single-view cone-beam X-ray
• Authors: Christiaan G.A. Viviers, Joel de Bruijn, Lena Filatova, Peter H.N. de With and Fons van der Sommen
• Abstract summary: 6D Object estimation based on deep learning models for X-ray images often use custom geometries that employ extensive CAD models and simulated data for training purposes. Recent RGB-based methods opt to solve estimation problems using small datasets, making them more attractive for the X-ray domain where medical data is scarcely available. We refine an existing RGB-based model (SingleShot) to estimate the 6D pose of a marked cube from X-ray images by creating a generic solution trained on only real X-ray data adjusted for X-ray geometry.
• Score: 6.971105483667455
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep learning-based pose estimation algorithms can successfully estimate the pose of objects in an image, especially in the field of color images. 6D Object pose estimation based on deep learning models for X-ray images often use custom architectures that employ extensive CAD models and simulated data for training purposes. Recent RGB-based methods opt to solve pose estimation problems using small datasets, making them more attractive for the X-ray domain where medical data is scarcely available. We refine an existing RGB-based model (SingleShotPose) to estimate the 6D pose of a marked cube from grayscale X-ray images by creating a generic solution trained on only real X-ray data and adjusted for X-ray acquisition geometry. The model regresses 2D control points and calculates the pose through 2D/3D correspondences using Perspective-n-Point(PnP), allowing a single trained model to be used across all supporting cone-beam-based X-ray geometries. Since modern X-ray systems continuously adjust acquisition parameters during a procedure, it is essential for such a pose estimation network to consider these parameters in order to be deployed successfully and find a real use case. With a 5-cm/5-degree accuracy of 93% and an average 3D rotation error of 2.2 degrees, the results of the proposed approach are comparable with state-of-the-art alternatives, while requiring significantly less real training examples and being applicable in real-time applications.

Related papers

• Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images [60.0898989456276]
  We present a new framework named Cas6D for few-shot 6DoF pose estimation that is generalizable and uses only RGB images. To address the false positives of target object detection in the extreme few-shot setting, our framework utilizes a self-supervised pre-trained ViT to learn robust feature representations. Experimental results on the LINEMOD and GenMOP datasets demonstrate that Cas6D outperforms state-of-the-art methods by 9.2% and 3.8% accuracy (Proj-5) under the 32-shot setting.
  arXiv  Detail & Related papers  (2023-06-13T07:45:42Z)
• 6D Object Pose Estimation from Approximate 3D Models for Orbital Robotics [19.64111218032901]
  We present a novel technique to estimate the 6D pose of objects from single images. We employ a dense 2D-to-3D correspondence predictor that regresses 3D model coordinates for every pixel. Our method achieves state-of-the-art performance on the SPEED+ dataset and has won the SPEC2021 post-mortem competition.
  arXiv  Detail & Related papers  (2023-03-23T13:18:05Z)
• Oral-3Dv2: 3D Oral Reconstruction from Panoramic X-Ray Imaging with Implicit Neural Representation [3.8215162658168524]
  Oral-3Dv2 is a non-adversarial-learning-based model in 3D radiology reconstruction from a single panoramic X-ray image. Our model learns to represent the 3D oral structure in an implicit way by mapping 2D coordinates into density values of voxels in the 3D space. To the best of our knowledge, this is the first work of a non-adversarial-learning-based model in 3D radiology reconstruction from a single panoramic X-ray image.
  arXiv  Detail & Related papers  (2023-03-21T18:17:27Z)
• 3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation [50.15926681475939]
  Inverse graphics aims to infer the 3D scene structure from 2D images. We introduce probabilistic modeling to quantify uncertainty and achieve robustness in 6D pose estimation tasks. 3DNEL effectively combines learned neural embeddings from RGB with depth information to improve robustness in sim-to-real 6D object pose estimation from RGB-D images.
  arXiv  Detail & Related papers  (2023-02-07T20:48:35Z)
• CNN-based real-time 2D-3D deformable registration from a single X-ray projection [2.1198879079315573]
  This paper presents a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image. A dataset composed of displacement fields and 2D projections of the anatomy is generated from a preoperative scan. A neural network is trained to recover the unknown 3D displacement field from a single projection image.
  arXiv  Detail & Related papers  (2022-12-15T09:57:19Z)
• Unseen Object 6D Pose Estimation: A Benchmark and Baselines [62.8809734237213]
  We propose a new task that enables and facilitates algorithms to estimate the 6D pose estimation of novel objects during testing. We collect a dataset with both real and synthetic images and up to 48 unseen objects in the test set. By training an end-to-end 3D correspondences network, our method finds corresponding points between an unseen object and a partial view RGBD image accurately and efficiently.
  arXiv  Detail & Related papers  (2022-06-23T16:29:53Z)
• Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [64.7198752089041]
  Given a set of known 3D objects and an RGB or RGB-D input image, we detect and estimate the 6D pose of each object. Our approach iteratively refines both pose and correspondence in a tightly coupled manner, allowing us to dynamically remove outliers to improve accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:00:08Z)
• NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation [44.42449011619408]
  We present a weakly-supervised reconstruction-based pipeline, named NeRF-Pose, which needs only 2D object segmentation and known relative camera poses during training. A NeRF-enabled RAN+SAC algorithm is used to estimate stable and accurate pose from the predicted correspondences. Experiments on LineMod-Occlusion show that the proposed method has state-of-the-art accuracy in comparison to the best 6D pose estimation methods.
  arXiv  Detail & Related papers  (2022-03-09T15:28:02Z)
• GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation [71.83992173720311]
  6D pose estimation from a single RGB image is a fundamental task in computer vision. We propose a simple yet effective Geometry-guided Direct Regression Network (GDR-Net) to learn the 6D pose in an end-to-end manner. Our approach remarkably outperforms state-of-the-art methods on LM, LM-O and YCB-V datasets.
  arXiv  Detail & Related papers  (2021-02-24T09:11:31Z)
• Deep Learning compatible Differentiable X-ray Projections for Inverse Rendering [8.926091372824942]
  We propose a differentiable by deriving the distance travelled by a ray inside mesh structures to generate a distance map. We show its application by solving the inverse problem, namely reconstructing 3D models from real 2D fluoroscopy images of the pelvis.
  arXiv  Detail & Related papers  (2021-02-04T22:06:05Z)
• Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation [62.618227434286]
  We present a novel learning approach to recover the 6D poses and sizes of unseen object instances from an RGB-D image. We propose a deep network to reconstruct the 3D object model by explicitly modeling the deformation from a pre-learned categorical shape prior.
  arXiv  Detail & Related papers  (2020-07-16T16:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.