On Self-Contact and Human Pose

• URL: http://arxiv.org/abs/2104.03176v2
• Date: Thu, 8 Apr 2021 07:29:50 GMT
• Title: On Self-Contact and Human Pose
• Authors: Lea M\"uller and Ahmed A. A. Osman and Siyu Tang and Chun-Hao P. Huang and Michael J. Black
• Abstract summary: We develop new datasets and methods that significantly improve human pose estimation with self-contact. We show that the new self-contact training data significantly improves 3D human pose estimates on withheld test data and existing datasets like 3DPW.
• Score: 50.96752167102025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: People touch their face 23 times an hour, they cross their arms and legs, put their hands on their hips, etc. While many images of people contain some form of self-contact, current 3D human pose and shape (HPS) regression methods typically fail to estimate this contact. To address this, we develop new datasets and methods that significantly improve human pose estimation with self-contact. First, we create a dataset of 3D Contact Poses (3DCP) containing SMPL-X bodies fit to 3D scans as well as poses from AMASS, which we refine to ensure good contact. Second, we leverage this to create the Mimic-The-Pose (MTP) dataset of images, collected via Amazon Mechanical Turk, containing people mimicking the 3DCP poses with selfcontact. Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images. Fourth, for more image variety, we label a dataset of in-the-wild images with Discrete Self-Contact (DSC) information and use another new optimization method, SMPLify-DC, that exploits discrete contacts during pose optimization. Finally, we use our datasets during SPIN training to learn a new 3D human pose regressor, called TUCH (Towards Understanding Contact in Humans). We show that the new self-contact training data significantly improves 3D human pose estimates on withheld test data and existing datasets like 3DPW. Not only does our method improve results for self-contact poses, but it also improves accuracy for non-contact poses. The code and data are available for research purposes at https://tuch.is.tue.mpg.de.

Related papers

• CameraHMR: Aligning People with Perspective [54.05758012879385]
  We address the challenge of accurate 3D human pose and shape estimation from monocular images. Existing training datasets containing real images with pseudo ground truth (pGT) use SMPLify to fit SMPL to sparse 2D joint locations. We make two contributions that improve pGT accuracy.
  arXiv  Detail & Related papers  (2024-11-12T19:12:12Z)
• MPL: Lifting 3D Human Pose from Multi-view 2D Poses [75.26416079541723]
  We propose combining 2D pose estimation, for which large and rich training datasets exist, and 2D-to-3D pose lifting, using a transformer-based network. Our experiments demonstrate decreases up to 45% in MPJPE errors compared to the 3D pose obtained by triangulating the 2D poses.
  arXiv  Detail & Related papers  (2024-08-20T12:55:14Z)
• Detecting Human-Object Contact in Images [75.35017308643471]
  Humans constantly contact objects to move and perform tasks. There exists no robust method to detect contact between the body and the scene from an image. We build a new dataset of human-object contacts for images.
  arXiv  Detail & Related papers  (2023-03-06T18:56:26Z)
• AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation [24.009674750548303]
  Testing a pre-trained 3D pose estimator on a new dataset results in a major performance drop. We propose AdaptPose, an end-to-end framework that generates synthetic 3D human motions from a source dataset. Our method outperforms previous work in cross-dataset evaluations by 14% and previous semi-supervised learning methods that use partial 3D annotations by 16%.
  arXiv  Detail & Related papers  (2021-12-22T00:27:52Z)
• Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency [55.94908688207493]
  We propose a self-supervised framework named SPICE that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. SPICE achieves state-of-the-art performance on the DeepFashion dataset.
  arXiv  Detail & Related papers  (2021-10-11T17:48:50Z)
• AGORA: Avatars in Geography Optimized for Regression Analysis [35.22486186509372]
  AGORA is a synthetic dataset with high realism and highly accurate ground truth. We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans. We evaluate existing state-of-the-art methods for 3D human pose estimation on this dataset and find that most methods perform poorly on images of children.
  arXiv  Detail & Related papers  (2021-04-29T20:33:25Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• Learning Complex 3D Human Self-Contact [33.83748199524761]
  Existing 3d reconstruction methods do not focus on body regions in self-contact. We develop a model for Self-Contact Prediction that estimates the body surface signature of self-contact. We show how more expressive 3d reconstructions can be recovered under self-contact signature constraints.
  arXiv  Detail & Related papers  (2020-12-18T17:09:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.