KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations

• URL: http://arxiv.org/abs/2002.12687v6
• Date: Fri, 7 Aug 2020 02:07:56 GMT
• Title: KeypointNet: A Large-scale 3D Keypoint Dataset Aggregated from Numerous Human Annotations
• Authors: Yang You, Yujing Lou, Chengkun Li, Zhoujun Cheng, Liangwei Li, Lizhuang Ma, Weiming Wang, Cewu Lu
• Abstract summary: KeypointNet is the first large-scale and diverse 3D keypoint dataset. It contains 103,450 keypoints and 8,234 3D models from 16 object categories. Ten state-of-the-art methods are benchmarked on our proposed dataset.
• Score: 56.34297279246823
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting 3D objects keypoints is of great interest to the areas of both graphics and computer vision. There have been several 2D and 3D keypoint datasets aiming to address this problem in a data-driven way. These datasets, however, either lack scalability or bring ambiguity to the definition of keypoints. Therefore, we present KeypointNet: the first large-scale and diverse 3D keypoint dataset that contains 103,450 keypoints and 8,234 3D models from 16 object categories, by leveraging numerous human annotations. To handle the inconsistency between annotations from different people, we propose a novel method to aggregate these keypoints automatically, through minimization of a fidelity loss. Finally, ten state-of-the-art methods are benchmarked on our proposed dataset. Our code and data are available on https://github.com/qq456cvb/KeypointNet.

Related papers

• Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features [20.935803672362283]
  We introduce an innovative unsupervised keypoint detector Key-Grid for both the rigid-body and deformable objects. We leverage the identified keypoint in formation to form a 3D grid feature heatmap called grid heatmap, which is used in the decoder section. Key-Grid achieves the state-of-the-art performance on the semantic consistency and position accuracy of keypoints.
  arXiv  Detail & Related papers  (2024-10-03T06:16:50Z)
• MoCap-to-Visual Domain Adaptation for Efficient Human Mesh Estimation from 2D Keypoints [8.405938712823563]
  Key2Mesh is a model that takes a set of 2D human pose keypoints as input and estimates the corresponding body mesh. Our results show that Key2Mesh sets the new state-of-the-art by outperforming other models in PA-MPJPE and 3DPW datasets.
  arXiv  Detail & Related papers  (2024-04-10T15:34:10Z)
• VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data [53.638818890966036]
  textitVoxelKP is a novel fully sparse network architecture tailored for human keypoint estimation in LiDAR data. We introduce sparse box-attention to focus on learning spatial correlations between keypoints within each human instance. We incorporate a spatial encoding to leverage absolute 3D coordinates when projecting 3D voxels to a 2D grid encoding a bird's eye view.
  arXiv  Detail & Related papers  (2023-12-11T23:50:14Z)
• Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features [64.39691149255717]
  Keypoint detection on 3D shapes requires semantic and geometric awareness while demanding high localization accuracy. We employ a keypoint candidate optimization module which aims to match the average observed distribution of keypoints on the shape. The resulting approach achieves a new state of the art for few-shot keypoint detection on the KeyPointNet dataset.
  arXiv  Detail & Related papers  (2023-11-29T21:58:41Z)
• DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching [14.837075102089]
  Keypoint detection is a pivotal step in 3D reconstruction, whereby sets of (up to) K points are detected in each view of a scene. Previous learning-based methods typically learn descriptors with keypoints, and treat the keypoint detection as a binary classification task on mutual nearest neighbours. In this work, we learn keypoints directly from 3D consistency. To this end, we derive a semi-supervised two-view detection objective to expand this set to a desired number of detections. Results show that our approach, DeDoDe, achieves significant gains on multiple geometry benchmarks.
  arXiv  Detail & Related papers  (2023-08-16T16:37:02Z)
• SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data [17.471342278936365]
  We propose a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2023-08-10T08:10:01Z)
• SNAKE: Shape-aware Neural 3D Keypoint Field [62.91169625183118]
  Detecting 3D keypoints from point clouds is important for shape reconstruction. This work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? We propose a novel unsupervised paradigm named SNAKE, which is short for shape-aware neural 3D keypoint field.
  arXiv  Detail & Related papers  (2022-06-03T17:58:43Z)
• End-to-End Learning of Multi-category 3D Pose and Shape Estimation [128.881857704338]
  We propose an end-to-end method that simultaneously detects 2D keypoints from an image and lifts them to 3D. The proposed method learns both 2D detection and 3D lifting only from 2D keypoints annotations. In addition to being end-to-end in image to 3D learning, our method also handles objects from multiple categories using a single neural network.
  arXiv  Detail & Related papers  (2021-12-19T17:10:40Z)
• Understanding Pixel-level 2D Image Semantics with 3D Keypoint Knowledge Engine [56.09471066808409]
  We propose a new method on predicting image corresponding semantics in 3D domain and then projecting them back onto 2D images to achieve pixel-level understanding. We build a large scale keypoint knowledge engine called KeypointNet, which contains 103,450 keypoints and 8,234 3D models from 16 object categories.
  arXiv  Detail & Related papers  (2021-11-21T13:25:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.