Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features

• URL: http://arxiv.org/abs/2410.02237v2
• Date: Wed, 16 Oct 2024 06:28:58 GMT
• Title: Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features
• Authors: Chengkai Hou, Zhengrong Xue, Bingyang Zhou, Jinghan Ke, Lin Shao, Huazhe Xu,
• Abstract summary: We introduce an innovative unsupervised keypoint detector Key-Grid for both the rigid-body and deformable objects. We leverage the identified keypoint in formation to form a 3D grid feature heatmap called grid heatmap, which is used in the decoder section. Key-Grid achieves the state-of-the-art performance on the semantic consistency and position accuracy of keypoints.
• Score: 20.935803672362283
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Detecting 3D keypoints with semantic consistency is widely used in many scenarios such as pose estimation, shape registration and robotics. Currently, most unsupervised 3D keypoint detection methods focus on the rigid-body objects. However, when faced with deformable objects, the keypoints they identify do not preserve semantic consistency well. In this paper, we introduce an innovative unsupervised keypoint detector Key-Grid for both the rigid-body and deformable objects, which is an autoencoder framework. The encoder predicts keypoints and the decoder utilizes the generated keypoints to reconstruct the objects. Unlike previous work, we leverage the identified keypoint in formation to form a 3D grid feature heatmap called grid heatmap, which is used in the decoder section. Grid heatmap is a novel concept that represents the latent variables for grid points sampled uniformly in the 3D cubic space, where these variables are the shortest distance between the grid points and the skeleton connected by keypoint pairs. Meanwhile, we incorporate the information from each layer of the encoder into the decoder section. We conduct an extensive evaluation of Key-Grid on a list of benchmark datasets. Key-Grid achieves the state-of-the-art performance on the semantic consistency and position accuracy of keypoints. Moreover, we demonstrate the robustness of Key-Grid to noise and downsampling. In addition, we achieve SE-(3) invariance of keypoints though generalizing Key-Grid to a SE(3)-invariant backbone.

Related papers

• KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation [87.23575166061413]
  KP-RED is a unified KeyPoint-driven REtrieval and Deformation framework. It takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models.
  arXiv  Detail & Related papers  (2024-03-15T08:44:56Z)
• CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [66.24726878647543]
  Estimating the 6-DoF pose of a rigid object from a single RGB image is a crucial yet challenging task. Recent studies have shown the great potential of dense correspondence-based solutions. We propose a novel pose estimation algorithm named CheckerPose, which improves on three main aspects.
  arXiv  Detail & Related papers  (2023-03-29T17:30:53Z)
• SNAKE: Shape-aware Neural 3D Keypoint Field [62.91169625183118]
  Detecting 3D keypoints from point clouds is important for shape reconstruction. This work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? We propose a novel unsupervised paradigm named SNAKE, which is short for shape-aware neural 3D keypoint field.
  arXiv  Detail & Related papers  (2022-06-03T17:58:43Z)
• Unsupervised Learning of Visual 3D Keypoints for Control [104.92063943162896]
  Learning sensorimotor control policies from high-dimensional images crucially relies on the quality of the underlying visual representations. We propose a framework to learn such a 3D geometric structure directly from images in an end-to-end unsupervised manner. These discovered 3D keypoints tend to meaningfully capture robot joints as well as object movements in a consistent manner across both time and 3D space.
  arXiv  Detail & Related papers  (2021-06-14T17:59:59Z)
• KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control [64.46042014759671]
  KeypointDeformer is an unsupervised method for shape control through automatically discovered 3D keypoints. Our approach produces intuitive and semantically consistent control of shape deformations.
  arXiv  Detail & Related papers  (2021-04-22T17:59:08Z)
• Skeleton Merger: an Unsupervised Aligned Keypoint Detector [44.983569951041]
  Skeleton Merger is an unsupervised aligned keypoint detector based on an Autoencoder architecture. It is capable of detecting semantically-rich salient keypoints with good alignment and shows comparable performance to supervised methods on the KeypointNet dataset.
  arXiv  Detail & Related papers  (2021-03-19T14:00:39Z)
• UKPGAN: A General Self-Supervised Keypoint Detector [43.35270822722044]
  UKPGAN is a general self-supervised 3D keypoint detector. Our keypoints align well with human annotated keypoint labels. Our model is stable under both rigid and non-rigid transformations.
  arXiv  Detail & Related papers  (2020-11-24T09:08:21Z)
• KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations [56.34297279246823]
  KeypointNet is the first large-scale and diverse 3D keypoint dataset. It contains 103,450 keypoints and 8,234 3D models from 16 object categories. Ten state-of-the-art methods are benchmarked on our proposed dataset.
  arXiv  Detail & Related papers  (2020-02-28T12:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.