Whole-Body Human Pose Estimation in the Wild

• URL: http://arxiv.org/abs/2007.11858v1
• Date: Thu, 23 Jul 2020 08:35:26 GMT
• Title: Whole-Body Human Pose Estimation in the Wild
• Authors: Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo
• Abstract summary: COCO-WholeBody extends COCO dataset with whole-body annotations. It is the first benchmark that has manual annotations on the entire human body. A single-network model, named ZoomNet, is devised to take into account the hierarchical structure of the full human body.
• Score: 88.09875133989155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates the task of 2D human whole-body pose estimation, which aims to localize dense landmarks on the entire human body including face, hands, body, and feet. As existing datasets do not have whole-body annotations, previous methods have to assemble different deep models trained independently on different datasets of the human face, hand, and body, struggling with dataset biases and large model complexity. To fill in this blank, we introduce COCO-WholeBody which extends COCO dataset with whole-body annotations. To our best knowledge, it is the first benchmark that has manual annotations on the entire human body, including 133 dense landmarks with 68 on the face, 42 on hands and 23 on the body and feet. A single-network model, named ZoomNet, is devised to take into account the hierarchical structure of the full human body to solve the scale variation of different body parts of the same person. ZoomNet is able to significantly outperform existing methods on the proposed COCO-WholeBody dataset. Extensive experiments show that COCO-WholeBody not only can be used to train deep models from scratch for whole-body pose estimation but also can serve as a powerful pre-training dataset for many different tasks such as facial landmark detection and hand keypoint estimation. The dataset is publicly available at https://github.com/jin-s13/COCO-WholeBody.

Related papers

• MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction [12.942635715952525]
  Multiple cameras can provide comprehensive multi-view video coverage of a person. Previous studies have overlooked the challenges posed by self-occlusion under multiple views. We introduce a method to reconstruct the 3D human body from multiple uncalibrated camera views.
  arXiv  Detail & Related papers  (2024-03-08T05:03:25Z)
• Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats [80.12253291709673]
  We propose a novel affine-combining autoencoder (ACAE) method to perform dimensionality reduction on the number of landmarks. Our approach scales to an extreme multi-dataset regime, where we use 28 3D human pose datasets to supervise one model.
  arXiv  Detail & Related papers  (2022-12-29T22:22:49Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)
• H3WB: Human3.6M 3D WholeBody Dataset and Benchmark [15.472137969924457]
  We present a benchmark for 3D human whole-body pose estimation. Currently, the lack of a fully annotated and accurate 3D whole-body dataset results in deep networks being trained separately on specific body parts. We introduce the Human3.6M 3D WholeBody dataset, which provides whole-body annotations for the Human3.6M dataset.
  arXiv  Detail & Related papers  (2022-11-28T19:00:02Z)
• ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild [97.0025378036642]
  We propose a single-network approach, termed ZoomNet, to take into account the hierarchical structure of the full human body. We also propose a neural architecture search framework, termed ZoomNAS, to promote both the accuracy and efficiency of whole-body pose estimation. To train and evaluate ZoomNAS, we introduce the first large-scale 2D human whole-body dataset, namely COCO-WholeBody V1.0.
  arXiv  Detail & Related papers  (2022-08-23T16:33:57Z)
• Learning Visibility for Robust Dense Human Body Estimation [78.37389398573882]
  Estimating 3D human pose and shape from 2D images is a crucial yet challenging task. We learn dense human body estimation that is robust to partial observations. We obtain pseudo ground-truths of visibility labels from dense UV correspondences and train a neural network to predict visibility along with 3D coordinates.
  arXiv  Detail & Related papers  (2022-08-23T00:01:05Z)
• HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation [13.198689566654108]
  We present a new bottom-up one-stage method for whole-body pose estimation. We build a hierarchical point representation of body parts and jointly regress them. On the COCO WholeBody dataset, HPRNet significantly outperforms all previous bottom-up methods.
  arXiv  Detail & Related papers  (2021-06-08T11:56:38Z)
• Monocular Real-time Full Body Capture with Inter-part Correlations [66.22835689189237]
  We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image. Our approach uses a new neural network architecture that exploits correlations between body and hands at high computational efficiency.
  arXiv  Detail & Related papers  (2020-12-11T02:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.