Collaborative Regression of Expressive Bodies using Moderation

• URL: http://arxiv.org/abs/2105.05301v1
• Date: Tue, 11 May 2021 18:55:59 GMT
• Title: Collaborative Regression of Expressive Bodies using Moderation
• Authors: Yao Feng, Vasileios Choutas, Timo Bolkart, Dimitrios Tzionas, Michael J. Black
• Abstract summary: Methods that estimate 3D bodies, faces, or hands have progressed significantly, yet separately. We introduce PIXIE, which produces animatable, whole-body 3D avatars from a single image. We label training images as male, female, or non-binary, and train PIXIE to infer "gendered" 3D body shapes with a novel shape loss.
• Score: 54.730550151409474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering expressive humans from images is essential for understanding human behavior. Methods that estimate 3D bodies, faces, or hands have progressed significantly, yet separately. Face methods recover accurate 3D shape and geometric details, but need a tight crop and struggle with extreme views and low resolution. Whole-body methods are robust to a wide range of poses and resolutions, but provide only a rough 3D face shape without details like wrinkles. To get the best of both worlds, we introduce PIXIE, which produces animatable, whole-body 3D avatars from a single image, with realistic facial detail. To get accurate whole bodies, PIXIE uses two key observations. First, body parts are correlated, but existing work combines independent estimates from body, face, and hand experts, by trusting them equally. PIXIE introduces a novel moderator that merges the features of the experts, weighted by their confidence. Uniquely, part experts can contribute to the whole, using SMPL-X's shared shape space across all body parts. Second, human shape is highly correlated with gender, but existing work ignores this. We label training images as male, female, or non-binary, and train PIXIE to infer "gendered" 3D body shapes with a novel shape loss. In addition to 3D body pose and shape parameters, PIXIE estimates expression, illumination, albedo and 3D surface displacements for the face. Quantitative and qualitative evaluation shows that PIXIE estimates 3D humans with a more accurate whole-body shape and detailed face shape than the state of the art. Our models and code are available for research at https://pixie.is.tue.mpg.de.

Related papers

• Cloth2Body: Generating 3D Human Body Mesh from 2D Clothing [54.29207348918216]
  Cloth2Body needs to address new and emerging challenges raised by the partial observation of the input and the high diversity of the output. We propose an end-to-end framework that can accurately estimate 3D body mesh parameterized by pose and shape from a 2D clothing image. As shown by experimental results, the proposed framework achieves state-of-the-art performance and can effectively recover natural and diverse 3D body meshes from 2D images.
  arXiv  Detail & Related papers  (2023-09-28T06:18:38Z)
• ECON: Explicit Clothed humans Optimized via Normal integration [54.51948104460489]
  We present ECON, a method for creating 3D humans in loose clothes. It infers detailed 2D maps for the front and back side of a clothed person. It "inpaints" the missing geometry between d-BiNI surfaces.
  arXiv  Detail & Related papers  (2022-12-14T18:59:19Z)
• Accurate 3D Body Shape Regression using Metric and Semantic Attributes [55.58629009876271]
  We show that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes. This is the first demonstration that 3D body shape regression from images can be trained from easy-to-obtain anthropometric measurements and linguistic shape attributes.
  arXiv  Detail & Related papers  (2022-06-14T17:54:49Z)
• Single-view 3D Body and Cloth Reconstruction under Complex Poses [37.86174829271747]
  We extend existing implicit function-based models to deal with images of humans with arbitrary poses and self-occluded limbs. We learn an implicit function that maps the input image to a 3D body shape with a low level of detail. We then learn a displacement map, conditioned on the smoothed surface, which encodes the high-frequency details of the clothes and body.
  arXiv  Detail & Related papers  (2022-05-09T07:34:06Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• Monocular Expressive Body Regression through Body-Driven Attention [68.63766976089842]
  We introduce ExPose, which regresses the body, face, and hands, in SMPL-X format, from an RGB image. hands and faces are much smaller than the body, occupying very few image pixels. We observe that body estimation localizes the face and hands reasonably well.
  arXiv  Detail & Related papers  (2020-08-20T16:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.