SMPLy Benchmarking 3D Human Pose Estimation in the Wild

• URL: http://arxiv.org/abs/2012.02743v1
• Date: Fri, 4 Dec 2020 17:48:32 GMT
• Title: SMPLy Benchmarking 3D Human Pose Estimation in the Wild
• Authors: Vincent Leroy, Philippe Weinzaepfel, Romain Br\'egier, Hadrien Combaluzier, Gr\'egory Rogez
• Abstract summary: Mannequin Challenge dataset contains in-the-wild videos of people frozen in action like statues. A total of 24,428 frames with registered body models are then selected from 567 scenes at almost no cost. We benchmark state-of-the-art SMPL-based human pose estimation methods on this dataset.
• Score: 14.323219585166573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predicting 3D human pose from images has seen great recent improvements. Novel approaches that can even predict both pose and shape from a single input image have been introduced, often relying on a parametric model of the human body such as SMPL. While qualitative results for such methods are often shown for images captured in-the-wild, a proper benchmark in such conditions is still missing, as it is cumbersome to obtain ground-truth 3D poses elsewhere than in a motion capture room. This paper presents a pipeline to easily produce and validate such a dataset with accurate ground-truth, with which we benchmark recent 3D human pose estimation methods in-the-wild. We make use of the recently introduced Mannequin Challenge dataset which contains in-the-wild videos of people frozen in action like statues and leverage the fact that people are static and the camera moving to accurately fit the SMPL model on the sequences. A total of 24,428 frames with registered body models are then selected from 567 scenes at almost no cost, using only online RGB videos. We benchmark state-of-the-art SMPL-based human pose estimation methods on this dataset. Our results highlight that challenges remain, in particular for difficult poses or for scenes where the persons are partially truncated or occluded.

Related papers

• ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation [54.86887812687023]
  Most 3D-HPE methods rely on regression models, which assume a one-to-one mapping between inputs and outputs. We propose ManiPose, a novel manifold-constrained multi-hypothesis model capable of proposing multiple candidate 3D poses for each 2D input. Unlike previous multi-hypothesis approaches, our solution is completely supervised and does not rely on complex generative models.
  arXiv  Detail & Related papers  (2023-12-11T13:50:10Z)
• MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling [59.74064212110042]
  mpmcan handle multiple tasks including 3D human pose estimation, 3D pose estimation from cluded 2D pose, and 3D pose completion in a textocbfsingle framework. We conduct extensive experiments and ablation studies on several widely used human pose datasets and achieve state-of-the-art performance on MPI-INF-3DHP.
  arXiv  Detail & Related papers  (2023-06-29T10:30:00Z)
• Self-supervised 3D Human Pose Estimation from a Single Image [1.0878040851638]
  We propose a new self-supervised method for predicting 3D human body pose from a single image. The prediction network is trained from a dataset of unlabelled images depicting people in typical poses and a set of unpaired 2D poses.
  arXiv  Detail & Related papers  (2023-04-05T10:26:21Z)
• ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses [23.554957518485324]
  We propose an unsupervised approach that learns to predict a 3D human pose from a single image. We estimate the 3D pose that is most likely over random projections, with the likelihood estimated using normalizing flows on 2D poses. We outperform the state-of-the-art unsupervised human pose estimation methods on the benchmark datasets Human3.6M and MPI-INF-3DHP in many metrics.
  arXiv  Detail & Related papers  (2021-12-14T01:12:45Z)
• Adversarial Parametric Pose Prior [106.12437086990853]
  We learn a prior that restricts the SMPL parameters to values that produce realistic poses via adversarial training. We show that our learned prior covers the diversity of the real-data distribution, facilitates optimization for 3D reconstruction from 2D keypoints, and yields better pose estimates when used for regression from images.
  arXiv  Detail & Related papers  (2021-12-08T10:05:32Z)
• Heuristic Weakly Supervised 3D Human Pose Estimation [13.82540778667711]
  weakly supervised 3D human pose (HW-HuP) solution to estimate 3D poses in when no ground truth 3D pose data is available. We show that HW-HuP meaningfully improves upon state-of-the-art models in two practical settings where 3D pose data can hardly be obtained: human poses in bed, and infant poses in the wild.
  arXiv  Detail & Related papers  (2021-05-23T18:40:29Z)
• AGORA: Avatars in Geography Optimized for Regression Analysis [35.22486186509372]
  AGORA is a synthetic dataset with high realism and highly accurate ground truth. We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans. We evaluate existing state-of-the-art methods for 3D human pose estimation on this dataset and find that most methods perform poorly on images of children.
  arXiv  Detail & Related papers  (2021-04-29T20:33:25Z)
• 3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data [77.57798334776353]
  We consider the problem of obtaining dense 3D reconstructions of humans from single and partially occluded views. We suggest that ambiguities can be modelled more effectively by parametrizing the possible body shapes and poses. We show that our method outperforms alternative approaches in ambiguous pose recovery on standard benchmarks for 3D humans.
  arXiv  Detail & Related papers  (2020-11-02T13:55:31Z)
• Towards Generalization of 3D Human Pose Estimation In The Wild [73.19542580408971]
  3DBodyTex.Pose is a dataset that addresses the task of 3D human pose estimation in-the-wild. 3DBodyTex.Pose offers high quality and rich data containing 405 different real subjects in various clothing and poses, and 81k image samples with ground-truth 2D and 3D pose annotations.
  arXiv  Detail & Related papers  (2020-04-21T13:31:58Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.