Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wild

• URL: http://arxiv.org/abs/2103.10978v1
• Date: Fri, 19 Mar 2021 18:32:16 GMT
• Title: Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wild
• Authors: Akash Sengupta, Ignas Budvytis, Roberto Cipolla
• Abstract summary: We propose a new task: shape and pose estimation from a group of multiple images of a human subject. Our solution predicts distributions over SMPL body shape and pose parameters conditioned on the input images in the group. We show that the additional body shape information present in multi-image input groups improves 3D human shape estimation metrics.
• Score: 25.647676661390282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses the problem of 3D human body shape and pose estimation from RGB images. Recent progress in this field has focused on single images, video or multi-view images as inputs. In contrast, we propose a new task: shape and pose estimation from a group of multiple images of a human subject, without constraints on subject pose, camera viewpoint or background conditions between images in the group. Our solution to this task predicts distributions over SMPL body shape and pose parameters conditioned on the input images in the group. We probabilistically combine predicted body shape distributions from each image to obtain a final multi-image shape prediction. We show that the additional body shape information present in multi-image input groups improves 3D human shape estimation metrics compared to single-image inputs on the SSP-3D dataset and a private dataset of tape-measured humans. In addition, predicting distributions over 3D bodies allows us to quantify pose prediction uncertainty, which is useful when faced with challenging input images with significant occlusion. Our method demonstrates meaningful pose uncertainty on the 3DPW dataset and is competitive with the state-of-the-art in terms of pose estimation metrics.

Related papers

• ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation [54.86887812687023]
  Most 3D-HPE methods rely on regression models, which assume a one-to-one mapping between inputs and outputs. We propose ManiPose, a novel manifold-constrained multi-hypothesis model capable of proposing multiple candidate 3D poses for each 2D input. Unlike previous multi-hypothesis approaches, our solution is completely supervised and does not rely on complex generative models.
  arXiv  Detail & Related papers  (2023-12-11T13:50:10Z)
• HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation [27.14060158187953]
  Recent approaches predict a probability distribution over plausible 3D pose and shape parameters conditioned on the image. We show that these approaches exhibit a trade-off between three key properties. Our method, HuManiFlow, predicts simultaneously accurate, consistent and diverse distributions.
  arXiv  Detail & Related papers  (2023-05-11T16:49:19Z)
• Self-supervised 3D Human Pose Estimation from a Single Image [1.0878040851638]
  We propose a new self-supervised method for predicting 3D human body pose from a single image. The prediction network is trained from a dataset of unlabelled images depicting people in typical poses and a set of unpaired 2D poses.
  arXiv  Detail & Related papers  (2023-04-05T10:26:21Z)
• Learning Visibility for Robust Dense Human Body Estimation [78.37389398573882]
  Estimating 3D human pose and shape from 2D images is a crucial yet challenging task. We learn dense human body estimation that is robust to partial observations. We obtain pseudo ground-truths of visibility labels from dense UV correspondences and train a neural network to predict visibility along with 3D coordinates.
  arXiv  Detail & Related papers  (2022-08-23T00:01:05Z)
• Adversarial Parametric Pose Prior [106.12437086990853]
  We learn a prior that restricts the SMPL parameters to values that produce realistic poses via adversarial training. We show that our learned prior covers the diversity of the real-data distribution, facilitates optimization for 3D reconstruction from 2D keypoints, and yields better pose estimates when used for regression from images.
  arXiv  Detail & Related papers  (2021-12-08T10:05:32Z)
• Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model [25.647676661390282]
  This paper addresses the problem of 3D human body shape and pose estimation from RGB images. We present a method that predicts distributions over local body shape in the form of semantic body measurements. We show that our method outperforms the current state-of-the-art in terms of identity-dependent body shape estimation accuracy.
  arXiv  Detail & Related papers  (2021-11-30T13:50:45Z)
• Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild [25.647676661390282]
  This paper addresses the problem of 3D human body shape and pose estimation from an RGB image. We train a deep neural network to estimate a hierarchical matrix-Fisher distribution over relative 3D joint rotation matrices. We show that our method is competitive with the state-of-the-art in terms of 3D shape and pose metrics on the SSP-3D and 3DPW datasets.
  arXiv  Detail & Related papers  (2021-10-03T11:59:37Z)
• 3D Human Pose Regression using Graph Convolutional Network [68.8204255655161]
  We propose a graph convolutional network named PoseGraphNet for 3D human pose regression from 2D poses. Our model's performance is close to the state-of-the-art, but with much fewer parameters.
  arXiv  Detail & Related papers  (2021-05-21T14:41:31Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild [101.70320427145388]
  We propose a weakly-supervised approach that does not require 3D annotations and learns to estimate 3D poses from unlabeled multi-view data. We evaluate our proposed approach on two large scale datasets.
  arXiv  Detail & Related papers  (2020-03-17T08:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.