SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data

• URL: http://arxiv.org/abs/2308.05410v1
• Date: Thu, 10 Aug 2023 08:10:01 GMT
• Title: SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data
• Authors: Mohammad Zohaib and Alessio Del Bue
• Abstract summary: We propose a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches.
• Score: 17.471342278936365
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. Our proposed model adheres to the following principles: i) keypoints inference is fully unsupervised (no annotation given), ii) keypoints position error should be low and resilient to PCD perturbations (robustness), iii) keypoints should not change their indexes for the intra-class objects (semantic coherence), iv) keypoints should be close to or proximal to PCD surface (compactness). We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation that does not assume any a priori knowledge of the object class, and a model architecture with coupled auxiliary losses that promotes the desired keypoints properties. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches. The experiments show that our approach outperforms by estimating keypoints with improved coverage (+9.41%) while being semantically consistent (+4.66%) that best characterizes the object's 3D shape for downstream tasks. Code and data are available at: https://github.com/IITPAVIS/SC3K

Related papers

• Visibility-Aware Keypoint Localization for 6DoF Object Pose Estimation [56.07676459156789]
  Localizing 3D keypoints in a 2D image is an effective way to establish 3D-2D correspondences for 6DoF object pose estimation. In this paper, we address this issue by localizing the important keypoints in terms of visibility. We construct VAPO (Visibility-Aware POse estimator) by integrating the visibility-aware importance with a state-of-the-art pose estimation algorithm.
  arXiv  Detail & Related papers  (2024-03-21T16:59:45Z)
• Meta-Point Learning and Refining for Category-Agnostic Pose Estimation [46.98479393474727]
  Category-agnostic pose estimation (CAPE) aims to predict keypoints for arbitrary classes given a few support images annotated with keypoints. We propose a novel framework for CAPE based on such potential keypoints (named as meta-points)
  arXiv  Detail & Related papers  (2024-03-20T14:54:33Z)
• Learning Better Keypoints for Multi-Object 6DoF Pose Estimation [1.0878040851638]
  We train a graph network to select a set of disperse keypoints with similarly distributed votes. These votes, learned by a regression network to accumulate evidence for the keypoint locations, can be regressed more accurately. Experiments demonstrate the keypoints selected by KeyGNet improved the accuracy for all evaluation metrics of all seven datasets tested.
  arXiv  Detail & Related papers  (2023-08-15T15:11:13Z)
• Object Localization under Single Coarse Point Supervision [107.46800858130658]
  We propose a POL method using coarse point annotations, relaxing the supervision signals from accurate key points to freely spotted points. CPR constructs point bags, selects semantic-correlated points, and produces semantic center points through multiple instance learning (MIL) In this way, CPR defines a weakly supervised evolution procedure, which ensures training high-performance object localizer under coarse point supervision.
  arXiv  Detail & Related papers  (2022-03-17T14:14:11Z)
• SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection [78.90102636266276]
  We propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA) Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection.
  arXiv  Detail & Related papers  (2022-01-06T08:54:47Z)
• UKPGAN: A General Self-Supervised Keypoint Detector [43.35270822722044]
  UKPGAN is a general self-supervised 3D keypoint detector. Our keypoints align well with human annotated keypoint labels. Our model is stable under both rigid and non-rigid transformations.
  arXiv  Detail & Related papers  (2020-11-24T09:08:21Z)
• Keypoint Autoencoders: Learning Interest Points of Semantics [4.551313396927381]
  We propose Keypoint Autoencoder, an unsupervised learning method for detecting keypoints. We encourage selecting sparse semantic keypoints by enforcing the reconstruction from keypoints to the original point cloud. A downstream task of classifying shape with sparse keypoints is conducted to demonstrate the distinctiveness of our selected keypoints.
  arXiv  Detail & Related papers  (2020-08-11T03:43:18Z)
• A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds [54.73161039445703]
  We propose a novel self-training approach that enables a typical object detector trained only with point-level annotations. During training, we utilize the available point annotations to supervise the estimation of the center points of objects. Experimental results show that our approach significantly outperforms state-of-the-art point-supervised methods under both detection and counting tasks.
  arXiv  Detail & Related papers  (2020-07-25T02:14:42Z)
• Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation [85.96410825961966]
  We argue that the image features extracted at a central point contain limited information for predicting distant keypoints or bounding box boundaries. To facilitate inference, we propose to instead perform regression from a set of points placed at more advantageous positions. We apply this proposed framework, called Point-Set Anchors, to object detection, instance segmentation, and human pose estimation.
  arXiv  Detail & Related papers  (2020-07-06T15:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.