RelPose++: Recovering 6D Poses from Sparse-view Observations

• URL: http://arxiv.org/abs/2305.04926v2
• Date: Mon, 18 Dec 2023 15:49:22 GMT
• Title: RelPose++: Recovering 6D Poses from Sparse-view Observations
• Authors: Amy Lin, Jason Y. Zhang, Deva Ramanan, Shubham Tulsiani
• Abstract summary: We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
• Score: 66.6922660401558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images). This task is a vital pre-processing stage for nearly all contemporary (neural) reconstruction algorithms but remains challenging given sparse views, especially for objects with visual symmetries and texture-less surfaces. We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. We extend this approach in two key ways; first, we use attentional transformer layers to process multiple images jointly, since additional views of an object may resolve ambiguous symmetries in any given image pair (such as the handle of a mug that becomes visible in a third view). Second, we augment this network to also report camera translations by defining an appropriate coordinate system that decouples the ambiguity in rotation estimation from translation prediction. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories and also enables pose estimation and 3D reconstruction for in-the-wild objects.

Related papers

• SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation [79.12683101131368]
  Category-level object pose estimation, aiming to predict the 6D pose and 3D size of objects from known categories, typically struggles with large intra-class shape variation. We present SecondPose, a novel approach integrating object-specific geometric features with semantic category priors from DINOv2.
  arXiv  Detail & Related papers  (2023-11-18T17:14:07Z)
• Pseudo Flow Consistency for Self-Supervised 6D Object Pose Estimation [14.469317161361202]
  We propose a 6D object pose estimation method that can be trained with pure RGB images without any auxiliary information. We evaluate our method on three challenging datasets and demonstrate that it outperforms state-of-the-art self-supervised methods significantly.
  arXiv  Detail & Related papers  (2023-08-19T13:52:18Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• MV6D: Multi-View 6D Pose Estimation on RGB-D Frames Using a Deep Point-wise Voting Network [14.754297065772676]
  We present a novel multi-view 6D pose estimation method called MV6D. We base our approach on the PVN3D network that uses a single RGB-D image to predict keypoints of the target objects. In contrast to current multi-view pose detection networks such as CosyPose, our MV6D can learn the fusion of multiple perspectives in an end-to-end manner.
  arXiv  Detail & Related papers  (2022-08-01T23:34:43Z)
• Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [64.7198752089041]
  Given a set of known 3D objects and an RGB or RGB-D input image, we detect and estimate the 6D pose of each object. Our approach iteratively refines both pose and correspondence in a tightly coupled manner, allowing us to dynamically remove outliers to improve accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:00:08Z)
• Weakly Supervised Learning of Keypoints for 6D Object Pose Estimation [73.40404343241782]
  We propose a weakly supervised 6D object pose estimation approach based on 2D keypoint detection. Our approach achieves comparable performance with state-of-the-art fully supervised approaches.
  arXiv  Detail & Related papers  (2022-03-07T16:23:47Z)
• Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo [71.59494156155309]
  Existing approaches for multi-view 3D pose estimation explicitly establish cross-view correspondences to group 2D pose detections from multiple camera views. We present our multi-view 3D pose estimation approach based on plane sweep stereo to jointly address the cross-view fusion and 3D pose reconstruction in a single shot.
  arXiv  Detail & Related papers  (2021-04-06T03:49:35Z)
• PrimA6D: Rotational Primitive Reconstruction for Enhanced and Robust 6D Pose Estimation [11.873744190924599]
  We introduce a rotational primitive prediction based 6D object pose estimation using a single image as an input. We leverage a Variational AutoEncoder (VAE) to learn this underlying primitive and its associated keypoints. When evaluated over public datasets, our method yields a notable improvement over LINEMOD, Occlusion LINEMOD, and the Y-induced dataset.
  arXiv  Detail & Related papers  (2020-06-14T03:55:42Z)
• A Pose Proposal and Refinement Network for Better Object Pose Estimation [0.5735035463793008]
  We present a novel, end-to-end 6D object pose estimation method that operates on RGB inputs. Our proposed pipeline outperforms state-of-the-art RGB-based methods with competitive runtime performance.
  arXiv  Detail & Related papers  (2020-04-11T23:13:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.