3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels

• URL: http://arxiv.org/abs/2306.04745v1
• Date: Wed, 7 Jun 2023 19:46:30 GMT
• Title: 3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
• Authors: Zhenzhen Weng, Alexander S. Gorban, Jingwei Ji, Mahyar Najibi, Yin Zhou, Dragomir Anguelov
• Abstract summary: GC-KPL is an approach for learning 3D human joint locations from point clouds without human labels. We show that by training on a large training set without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach.
• Score: 78.69095161350059
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training a 3D human keypoint detector from point clouds in a supervised manner requires large volumes of high quality labels. While it is relatively easy to capture large amounts of human point clouds, annotating 3D keypoints is expensive, subjective, error prone and especially difficult for long-tail cases (pedestrians with rare poses, scooterists, etc.). In this work, we propose GC-KPL - Geometry Consistency inspired Key Point Leaning, an approach for learning 3D human joint locations from point clouds without human labels. We achieve this by our novel unsupervised loss formulations that account for the structure and movement of the human body. We show that by training on a large training set from Waymo Open Dataset without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach. Further, the backbone benefits from the unsupervised training and is useful in downstream fewshot learning of keypoints, where fine-tuning on only 10 percent of the labeled training data gives comparable performance to fine-tuning on the entire set. We demonstrated that GC-KPL outperforms by a large margin over SoTA when trained on entire dataset and efficiently leverages large volumes of unlabeled data.

Related papers

• Unsupervised 3D Keypoint Discovery with Multi-View Geometry [104.76006413355485]
  We propose an algorithm that learns to discover 3D keypoints on human bodies from multiple-view images without supervision or labels. Our approach discovers more interpretable and accurate 3D keypoints compared to other state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2022-11-23T10:25:12Z)
• Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysis [33.31864436614945]
  We propose a novel pre-training method for 3D point cloud models. Our pre-training is self-supervised by a local pixel/point level correspondence loss and a global image/point cloud level loss. These improved models outperform existing state-of-the-art methods on various datasets and downstream tasks.
  arXiv  Detail & Related papers  (2022-10-28T05:23:03Z)
• COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation [16.072116380353393]
  COARSE3D is a novel architecture-agnostic contrastive learning strategy for 3D segmentation. We leverage a prototype memory bank capturing class-wise global dataset information efficiently into a small number of prototypes acting as keys. Experiments show we outperform baselines on three challenging real-world outdoor datasets, working with as low as 0.001% annotations.
  arXiv  Detail & Related papers  (2022-10-04T17:54:53Z)
• Point Discriminative Learning for Unsupervised Representation Learning on 3D Point Clouds [54.31515001741987]
  We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds. We achieve this by imposing a novel point discrimination loss on the middle level and global level point features. Our method learns powerful representations and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-08-04T15:11:48Z)
• Self-Supervised Pretraining of 3D Features on any Point-Cloud [40.26575888582241]
  We present a simple self-supervised pertaining method that can work with any 3D data without 3D registration. We evaluate our models on 9 benchmarks for object detection, semantic segmentation, and object classification, where they achieve state-of-the-art results and can outperform supervised pretraining.
  arXiv  Detail & Related papers  (2021-01-07T18:55:21Z)
• S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency [11.357804868755155]
  We advocate semantic 3D keypoints as a visual representation, and present a semi-supervised training objective. Unlike local texture-based approaches, our model integrates contextual information from a large area. We demonstrate that this ability to locate semantic keypoints enables high level scripting of human understandable behaviours.
  arXiv  Detail & Related papers  (2020-09-30T14:44:54Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)
• Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels [77.65554439859967]
  We propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage. Experiments are done on three public datasets with different degrees of weak supervision.
  arXiv  Detail & Related papers  (2020-04-08T16:14:41Z)
• Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds [67.0904905172941]
  We propose a weakly supervised approach to predict point-level results using weak labels on 3D point clouds. To the best of our knowledge, this is the first method that uses cloud-level weak labels on raw 3D space to train a point cloud semantic segmentation network.
  arXiv  Detail & Related papers  (2020-03-29T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.