Learning Better Keypoints for Multi-Object 6DoF Pose Estimation

• URL: http://arxiv.org/abs/2308.07827v2
• Date: Fri, 10 Nov 2023 02:42:47 GMT
• Title: Learning Better Keypoints for Multi-Object 6DoF Pose Estimation
• Authors: Yangzheng Wu and Michael Greenspan
• Abstract summary: We train a graph network to select a set of disperse keypoints with similarly distributed votes. These votes, learned by a regression network to accumulate evidence for the keypoint locations, can be regressed more accurately. Experiments demonstrate the keypoints selected by KeyGNet improved the accuracy for all evaluation metrics of all seven datasets tested.
• Score: 1.0878040851638
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We address the problem of keypoint selection, and find that the performance of 6DoF pose estimation methods can be improved when pre-defined keypoint locations are learned, rather than being heuristically selected as has been the standard approach. We found that accuracy and efficiency can be improved by training a graph network to select a set of disperse keypoints with similarly distributed votes. These votes, learned by a regression network to accumulate evidence for the keypoint locations, can be regressed more accurately compared to previous heuristic keypoint algorithms. The proposed KeyGNet, supervised by a combined loss measuring both Wasserstein distance and dispersion, learns the color and geometry features of the target objects to estimate optimal keypoint locations. Experiments demonstrate the keypoints selected by KeyGNet improved the accuracy for all evaluation metrics of all seven datasets tested, for three keypoint voting methods. The challenging Occlusion LINEMOD dataset notably improved ADD(S) by +16.4% on PVN3D, and all core BOP datasets showed an AR improvement for all objects, of between +1% and +21.5%. There was also a notable increase in performance when transitioning from single object to multiple object training using KeyGNet keypoints, essentially eliminating the SISO-MIMO gap for Occlusion LINEMOD.

Related papers

• GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring [9.322937309882022]
  Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method.
  arXiv  Detail & Related papers  (2024-08-30T09:39:59Z)
• KGpose: Keypoint-Graph Driven End-to-End Multi-Object 6D Pose Estimation via Point-Wise Pose Voting [0.0]
  KGpose is an end-to-end framework for 6D pose estimation of multiple objects. Our approach combines keypoint-based method with learnable pose regression through keypoint-graph'
  arXiv  Detail & Related papers  (2024-07-12T01:06:00Z)
• Visibility-Aware Keypoint Localization for 6DoF Object Pose Estimation [56.07676459156789]
  Localizing 3D keypoints in a 2D image is an effective way to establish 3D-2D correspondences for 6DoF object pose estimation. In this paper, we address this issue by localizing the important keypoints in terms of visibility. We construct VAPO (Visibility-Aware POse estimator) by integrating the visibility-aware importance with a state-of-the-art pose estimation algorithm.
  arXiv  Detail & Related papers  (2024-03-21T16:59:45Z)
• SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios [2.786599193929693]
  We propose a new 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net) At the keypoint prediction stage, we designe a robust 3D keypoints selection strategy to locate 3D keypoints even in highly occluded scenes. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. On public Sil'eane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%.
  arXiv  Detail & Related papers  (2024-03-14T12:08:44Z)
• SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data [17.471342278936365]
  We propose a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2023-08-10T08:10:01Z)
• Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network [52.29330138835208]
  Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
  arXiv  Detail & Related papers  (2023-07-04T02:50:44Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• Accurate Grid Keypoint Learning for Efficient Video Prediction [87.71109421608232]
  Keypoint-based video prediction methods can consume substantial computing resources in training and deployment. In this paper, we design a new grid keypoint learning framework, aiming at a robust and explainable intermediate keypoint representation for long-term efficient video prediction. Our method outperforms the state-ofthe-art video prediction methods while saves 98% more than computing resources.
  arXiv  Detail & Related papers  (2021-07-28T05:04:30Z)
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression [81.05772887221333]
  We study the dense keypoint regression framework that is previously inferior to the keypoint detection and grouping framework. We present a simple yet effective approach, named disentangled keypoint regression (DEKR) We empirically show that the proposed direct regression method outperforms keypoint detection and grouping methods.
  arXiv  Detail & Related papers  (2021-04-06T05:54:46Z)
• Keypoint Autoencoders: Learning Interest Points of Semantics [4.551313396927381]
  We propose Keypoint Autoencoder, an unsupervised learning method for detecting keypoints. We encourage selecting sparse semantic keypoints by enforcing the reconstruction from keypoints to the original point cloud. A downstream task of classifying shape with sparse keypoints is conducted to demonstrate the distinctiveness of our selected keypoints.
  arXiv  Detail & Related papers  (2020-08-11T03:43:18Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.