H3WB: Human3.6M 3D WholeBody Dataset and Benchmark

• URL: http://arxiv.org/abs/2211.15692v2
• Date: Wed, 6 Sep 2023 12:22:24 GMT
• Title: H3WB: Human3.6M 3D WholeBody Dataset and Benchmark
• Authors: Yue Zhu, Nermin Samet, David Picard
• Abstract summary: We present a benchmark for 3D human whole-body pose estimation. Currently, the lack of a fully annotated and accurate 3D whole-body dataset results in deep networks being trained separately on specific body parts. We introduce the Human3.6M 3D WholeBody dataset, which provides whole-body annotations for the Human3.6M dataset.
• Score: 15.472137969924457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a benchmark for 3D human whole-body pose estimation, which involves identifying accurate 3D keypoints on the entire human body, including face, hands, body, and feet. Currently, the lack of a fully annotated and accurate 3D whole-body dataset results in deep networks being trained separately on specific body parts, which are combined during inference. Or they rely on pseudo-groundtruth provided by parametric body models which are not as accurate as detection based methods. To overcome these issues, we introduce the Human3.6M 3D WholeBody (H3WB) dataset, which provides whole-body annotations for the Human3.6M dataset using the COCO Wholebody layout. H3WB comprises 133 whole-body keypoint annotations on 100K images, made possible by our new multi-view pipeline. We also propose three tasks: i) 3D whole-body pose lifting from 2D complete whole-body pose, ii) 3D whole-body pose lifting from 2D incomplete whole-body pose, and iii) 3D whole-body pose estimation from a single RGB image. Additionally, we report several baselines from popular methods for these tasks. Furthermore, we also provide automated 3D whole-body annotations of TotalCapture and experimentally show that when used with H3WB it helps to improve the performance. Code and dataset is available at https://github.com/wholebody3d/wholebody3d

Related papers

• Decanus to Legatus: Synthetic training for 2D-3D human pose lifting [26.108023246654646]
  We propose an algorithm to generate infinite 3D synthetic human poses (Legatus) from a 3D pose distribution based on 10 initial handcrafted 3D poses (Decanus) Our results show that we can achieve 3D pose estimation performance comparable to methods using real data from specialized datasets but in a zero-shot setup, showing the potential of our framework.
  arXiv  Detail & Related papers  (2022-10-05T13:10:19Z)
• Accurate 3D Body Shape Regression using Metric and Semantic Attributes [55.58629009876271]
  We show that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes. This is the first demonstration that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes.
  arXiv  Detail & Related papers  (2022-06-14T17:54:49Z)
• Tracking People with 3D Representations [78.97070307547283]
  We present a novel approach for tracking multiple people in video. Unlike past approaches which employ 2D representations, we employ 3D representations of people, located in three-dimensional space. We find that 3D representations are more effective than 2D representations for tracking in these settings.
  arXiv  Detail & Related papers  (2021-11-15T16:15:21Z)
• Learning Temporal 3D Human Pose Estimation with Pseudo-Labels [3.0954251281114513]
  We present a simple, yet effective, approach for self-supervised 3D human pose estimation. We rely on triangulating 2D body pose estimates of a multiple-view camera system. Our method achieves state-of-the-art performance in the Human3.6M and MPI-INF-3DHP benchmarks.
  arXiv  Detail & Related papers  (2021-10-14T17:40:45Z)
• VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild [98.69191256693703]
  We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.
  arXiv  Detail & Related papers  (2021-08-05T08:35:44Z)
• Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation [70.23652933572647]
  Whole-body 3D human mesh estimation aims to reconstruct the 3D human body, hands, and face simultaneously. We present Hand4Whole, which has two strong points over previous works. Our Hand4Whole is trained in an end-to-end manner and produces much better 3D hand results than previous whole-body 3D human mesh estimation methods.
  arXiv  Detail & Related papers  (2020-11-23T16:48:35Z)
• GRAB: A Dataset of Whole-Body Human Grasping of Objects [53.00728704389501]
  Training computers to understand human grasping requires a rich dataset containing complex 3D object shapes, detailed contact information, hand pose and shape, and the 3D body motion over time. We collect a new dataset, called GRAB, of whole-body grasps, containing full 3D shape and pose sequences of 10 subjects interacting with 51 everyday objects of varying shape and size. This is a unique dataset, that goes well beyond existing ones for modeling and understanding how humans grasp and manipulate objects, how their full body is involved, and how interaction varies with the task.
  arXiv  Detail & Related papers  (2020-08-25T17:57:55Z)
• Towards Generalization of 3D Human Pose Estimation In The Wild [73.19542580408971]
  3DBodyTex.Pose is a dataset that addresses the task of 3D human pose estimation in-the-wild. 3DBodyTex.Pose offers high quality and rich data containing 405 different real subjects in various clothing and poses, and 81k image samples with ground-truth 2D and 3D pose annotations.
  arXiv  Detail & Related papers  (2020-04-21T13:31:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.