AGORA: Avatars in Geography Optimized for Regression Analysis

• URL: http://arxiv.org/abs/2104.14643v1
• Date: Thu, 29 Apr 2021 20:33:25 GMT
• Title: AGORA: Avatars in Geography Optimized for Regression Analysis
• Authors: Priyanka Patel, Chun-Hao P. Huang, Joachim Tesch, David T. Hoffmann, Shashank Tripathi, Michael J. Black
• Abstract summary: AGORA is a synthetic dataset with high realism and highly accurate ground truth. We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans. We evaluate existing state-of-the-art methods for 3D human pose estimation on this dataset and find that most methods perform poorly on images of children.
• Score: 35.22486186509372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While the accuracy of 3D human pose estimation from images has steadily improved on benchmark datasets, the best methods still fail in many real-world scenarios. This suggests that there is a domain gap between current datasets and common scenes containing people. To obtain ground-truth 3D pose, current datasets limit the complexity of clothing, environmental conditions, number of subjects, and occlusion. Moreover, current datasets evaluate sparse 3D joint locations corresponding to the major joints of the body, ignoring the hand pose and the face shape. To evaluate the current state-of-the-art methods on more challenging images, and to drive the field to address new problems, we introduce AGORA, a synthetic dataset with high realism and highly accurate ground truth. Here we use 4240 commercially-available, high-quality, textured human scans in diverse poses and natural clothing; this includes 257 scans of children. We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans, taking into account clothing. We create around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal. In total, AGORA consists of 173K individual person crops. We evaluate existing state-of-the-art methods for 3D human pose estimation on this dataset and find that most methods perform poorly on images of children. Hence, we extend the SMPL-X model to better capture the shape of children. Additionally, we fine-tune methods on AGORA and show improved performance on both AGORA and 3DPW, confirming the realism of the dataset. We provide all the registered 3D reference training data, rendered images, and a web-based evaluation site at https://agora.is.tue.mpg.de/.

Related papers

• AG3D: Learning to Generate 3D Avatars from 2D Image Collections [96.28021214088746]
  We propose a new adversarial generative model of realistic 3D people from 2D images. Our method captures shape and deformation of the body and loose clothing by adopting a holistic 3D generator. We experimentally find that our method outperforms previous 3D- and articulation-aware methods in terms of geometry and appearance.
  arXiv  Detail & Related papers  (2023-05-03T17:56:24Z)
• Decanus to Legatus: Synthetic training for 2D-3D human pose lifting [26.108023246654646]
  We propose an algorithm to generate infinite 3D synthetic human poses (Legatus) from a 3D pose distribution based on 10 initial handcrafted 3D poses (Decanus) Our results show that we can achieve 3D pose estimation performance comparable to methods using real data from specialized datasets but in a zero-shot setup, showing the potential of our framework.
  arXiv  Detail & Related papers  (2022-10-05T13:10:19Z)
• Accurate 3D Body Shape Regression using Metric and Semantic Attributes [55.58629009876271]
  We show that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes. This is the first demonstration that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes.
  arXiv  Detail & Related papers  (2022-06-14T17:54:49Z)
• Single-view 3D Body and Cloth Reconstruction under Complex Poses [37.86174829271747]
  We extend existing implicit function-based models to deal with images of humans with arbitrary poses and self-occluded limbs. We learn an implicit function that maps the input image to a 3D body shape with a low level of detail. We then learn a displacement map, conditioned on the smoothed surface, which encodes the high-frequency details of the clothes and body.
  arXiv  Detail & Related papers  (2022-05-09T07:34:06Z)
• DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance Fields for Articulated Avatars [92.37436369781692]
  We present DRaCoN, a framework for learning full-body volumetric avatars. It exploits the advantages of both the 2D and 3D neural rendering techniques. Experiments on the challenging ZJU-MoCap and Human3.6M datasets indicate that DRaCoN outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T17:59:15Z)
• Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency [55.94908688207493]
  We propose a self-supervised framework named SPICE that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. SPICE achieves state-of-the-art performance on the DeepFashion dataset.
  arXiv  Detail & Related papers  (2021-10-11T17:48:50Z)
• Heuristic Weakly Supervised 3D Human Pose Estimation [13.82540778667711]
  weakly supervised 3D human pose (HW-HuP) solution to estimate 3D poses in when no ground truth 3D pose data is available. We show that HW-HuP meaningfully improves upon state-of-the-art models in two practical settings where 3D pose data can hardly be obtained: human poses in bed, and infant poses in the wild.
  arXiv  Detail & Related papers  (2021-05-23T18:40:29Z)
• Neural 3D Clothes Retargeting from a Single Image [91.5030622330039]
  We present a method of clothes; generating the potential poses and deformations of a given 3D clothing template model to fit onto a person in a single RGB image. The problem is fundamentally ill-posed as attaining the ground truth data is impossible, i.e. images of people wearing the different 3D clothing template model model at exact same pose. We propose a semi-supervised learning framework that validates the physical plausibility of 3D deformation by matching with the prescribed body-to-cloth contact points and clothing to fit onto the unlabeled silhouette.
  arXiv  Detail & Related papers  (2021-01-29T20:50:34Z)
• SMPLy Benchmarking 3D Human Pose Estimation in the Wild [14.323219585166573]
  Mannequin Challenge dataset contains in-the-wild videos of people frozen in action like statues. A total of 24,428 frames with registered body models are then selected from 567 scenes at almost no cost. We benchmark state-of-the-art SMPL-based human pose estimation methods on this dataset.
  arXiv  Detail & Related papers  (2020-12-04T17:48:32Z)
• Towards Generalization of 3D Human Pose Estimation In The Wild [73.19542580408971]
  3DBodyTex.Pose is a dataset that addresses the task of 3D human pose estimation in-the-wild. 3DBodyTex.Pose offers high quality and rich data containing 405 different real subjects in various clothing and poses, and 81k image samples with ground-truth 2D and 3D pose annotations.
  arXiv  Detail & Related papers  (2020-04-21T13:31:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.