ZeroPose: CAD-Model-based Zero-Shot Pose Estimation
- URL: http://arxiv.org/abs/2305.17934v2
- Date: Mon, 21 Aug 2023 09:18:03 GMT
- Title: ZeroPose: CAD-Model-based Zero-Shot Pose Estimation
- Authors: Jianqiu Chen, Mingshan Sun, Tianpeng Bao, Rui Zhao, Liwei Wu, Zhenyu
He
- Abstract summary: We present a CAD model-based zero-shot pose estimation pipeline called ZeroPose.
The proposed method enables the accurate estimation of pose parameters for previously unseen objects without the need for training.
- Score: 19.495700754681124
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we present a CAD model-based zero-shot pose estimation
pipeline called ZeroPose. Existing pose estimation methods remain to require
expensive training when applied to an unseen object, which greatly hinders
their scalability in the practical application of industry. In contrast, the
proposed method enables the accurate estimation of pose parameters for
previously unseen objects without the need for training. Specifically, we
design a two-step pipeline consisting of CAD model-based zero-shot instance
segmentation and a zero-shot pose estimator. For the first step, there is a
simple but effective way to leverage CAD models and visual foundation models
SAM and Imagebind to segment the interest unseen object at the instance level.
For the second step, we based on the intensive geometric information in the CAD
model of the rigid object to propose a lightweight hierarchical geometric
structure matching mechanism achieving zero-shot pose estimation. Extensive
experimental results on the seven core datasets on the BOP challenge show that
the proposed zero-shot instance segmentation methods achieve comparable
performance with supervised MaskRCNN and the zero-shot pose estimation results
outperform the SOTA pose estimators with better efficiency.
Related papers
- NeuSurfEmb: A Complete Pipeline for Dense Correspondence-based 6D Object Pose Estimation without CAD Models [34.898217885820614]
We present a pipeline that does not require CAD models and allows training a state-of-the-art pose estimator requiring only a small set of real images as input.
Our method is based on a NeuS2 object representation, that we learn through a semi-automated procedure based on Structure-from-Motion (SfM) and object-agnostic segmentation.
We evaluate our method on the LINEMOD-Occlusion dataset, extensively studying the impact of its individual components and showing competitive performance with respect to approaches based on CAD models and PBR data.
arXiv Detail & Related papers (2024-07-16T22:48:22Z) - FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [55.77542145604758]
FoundationPose is a unified foundation model for 6D object pose estimation and tracking.
Our approach can be instantly applied at test-time to a novel object without fine-tuning.
arXiv Detail & Related papers (2023-12-13T18:28:09Z) - PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching [51.142988196855484]
We propose PoseMatcher, an accurate model free one-shot object pose estimator.
We create a new training pipeline for object to image matching based on a three-view system.
To enable PoseMatcher to attend to distinct input modalities, an image and a pointcloud, we introduce IO-Layer.
arXiv Detail & Related papers (2023-04-03T21:14:59Z) - OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD
Models [51.68715543630427]
OnePose relies on detecting repeatable image keypoints and is thus prone to failure on low-textured objects.
We propose a keypoint-free pose estimation pipeline to remove the need for repeatable keypoint detection.
A 2D-3D matching network directly establishes 2D-3D correspondences between the query image and the reconstructed point-cloud model.
arXiv Detail & Related papers (2023-01-18T17:47:13Z) - MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [84.80956484848505]
MegaPose is a method to estimate the 6D pose of novel objects, that is, objects unseen during training.
We present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects.
Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner.
arXiv Detail & Related papers (2022-12-13T19:30:03Z) - Semantic keypoint-based pose estimation from single RGB frames [64.80395521735463]
We present an approach to estimating the continuous 6-DoF pose of an object from a single RGB image.
The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model.
We show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios.
arXiv Detail & Related papers (2022-04-12T15:03:51Z) - Zero-Shot Category-Level Object Pose Estimation [24.822189326540105]
We tackle the problem of estimating the pose of novel object categories in a zero-shot manner.
This extends much of the existing literature by removing the need for pose-labelled datasets or category-specific CAD models.
Our method provides a six-fold improvement in average rotation accuracy at 30 degrees.
arXiv Detail & Related papers (2022-04-07T17:58:39Z) - Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object
Pose Estimation [30.04752448942084]
Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.
We propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.
arXiv Detail & Related papers (2021-10-30T06:46:44Z) - Spatial Attention Improves Iterative 6D Object Pose Estimation [52.365075652976735]
We propose a new method for 6D pose estimation refinement from RGB images.
Our main insight is that after the initial pose estimate, it is important to pay attention to distinct spatial features of the object.
We experimentally show that this approach learns to attend to salient spatial features and learns to ignore occluded parts of the object, leading to better pose estimation across datasets.
arXiv Detail & Related papers (2021-01-05T17:18:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.