Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation

• URL: http://arxiv.org/abs/2403.06164v2
• Date: Fri, 27 Sep 2024 12:08:07 GMT
• Title: Platypose: Calibrated Zero-Shot Multi-Hypothesis 3D Human Motion Estimation
• Authors: Paweł A. Pierzchlewicz, Caio O. da Silva, R. James Cotton, Fabian H. Sinz,
• Abstract summary: Platypose is a framework that uses a diffusion model pretrained on 3D human motion sequences for zero-shot 3D pose sequence estimation. Platypose achieves state-of-the-art calibration and competitive joint error when tested on static poses from Human3.6M, MPI-INF-3DHP and 3DPW.
• Score: 6.061037203796638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single camera 3D pose estimation is an ill-defined problem due to inherent ambiguities from depth, occlusion or keypoint noise. Multi-hypothesis pose estimation accounts for this uncertainty by providing multiple 3D poses consistent with the 2D measurements. Current research has predominantly concentrated on generating multiple hypotheses for single frame static pose estimation or single hypothesis motion estimation. In this study we focus on the new task of multi-hypothesis motion estimation. Multi-hypothesis motion estimation is not simply multi-hypothesis pose estimation applied to multiple frames, which would ignore temporal correlation across frames. Instead, it requires distributions which are capable of generating temporally consistent samples, which is significantly more challenging than multi-hypothesis pose estimation or single-hypothesis motion estimation. To this end, we introduce Platypose, a framework that uses a diffusion model pretrained on 3D human motion sequences for zero-shot 3D pose sequence estimation. Platypose outperforms baseline methods on multiple hypotheses for motion estimation. Additionally, Platypose also achieves state-of-the-art calibration and competitive joint error when tested on static poses from Human3.6M, MPI-INF-3DHP and 3DPW. Finally, because it is zero-shot, our method generalizes flexibly to different settings such as multi-camera inference.

Related papers

• Diffusion-based Pose Refinement and Muti-hypothesis Generation for 3D Human Pose Estimaiton [27.708016152889787]
  Previous probabilistic models for 3D Human Pose Estimation (3DHPE) aimed to enhance pose accuracy by generating multiple hypotheses. Most of the hypotheses generated deviate substantially from the true pose. Compared to deterministic models, the excessive uncertainty in probabilistic models leads to weaker performance in single-hypothesis prediction. We propose a diffusion-based refinement framework called DRPose, which refines the output of deterministic models by reverse diffusion.
  arXiv  Detail & Related papers  (2024-01-10T04:07:50Z)
• D3PRefiner: A Diffusion-based Denoise Method for 3D Human Pose Refinement [3.514184876338779]
  A Diffusion-based 3D Pose Refiner is proposed to refine the output of any existing 3D pose estimator. We leverage the architecture of current diffusion models to convert the distribution of noisy 3D poses into ground truth 3D poses. Experimental results demonstrate the proposed architecture can significantly improve the performance of current sequence-to-sequence 3D pose estimators.
  arXiv  Detail & Related papers  (2024-01-08T14:21:02Z)
• Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation [64.874000550443]
  A Diffusion-based 3D Pose estimation (D3DP) method with Joint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposed. The proposed JPMA assembles multiple hypotheses generated by D3DP into a single 3D pose for practical use. Our method outperforms the state-of-the-art deterministic and probabilistic approaches by 1.5% and 8.9%, respectively.
  arXiv  Detail & Related papers  (2023-03-21T04:00:47Z)
• DiffPose: Multi-hypothesis Human Pose Estimation using Diffusion models [5.908471365011943]
  We propose emphDiffPose, a conditional diffusion model that predicts multiple hypotheses for a given input image. We show that DiffPose slightly improves upon the state of the art for multi-hypothesis pose estimation for simple poses and outperforms it by a large margin for highly ambiguous poses.
  arXiv  Detail & Related papers  (2022-11-29T18:55:13Z)
• KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints [11.912058874826549]
  We propose a technique to infer joint orientations throughout a biomechanically informed, and subject-specific kinematic chain. We generate 3D pose motion sequences to assess the IK approach both for general accuracy, and accuracy in boundary cases. Our temporal algorithm achieves 6DOF pose estimates with low Mean Per Joint Angular Separation (MPJAS) errors.
  arXiv  Detail & Related papers  (2022-07-26T12:17:07Z)
• Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation [63.199549837604444]
  3D human pose estimation approaches leverage different forms of strong (2D/3D pose) or weak (multi-view or depth) paired supervision. We cast 3D pose learning as a self-supervised adaptation problem that aims to transfer the task knowledge from a labeled source domain to a completely unpaired target. We evaluate different self-adaptation settings and demonstrate state-of-the-art 3D human pose estimation performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-05T03:52:57Z)
• MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation [88.73883883964048]
  Estimating 3D human poses from monocular videos is a challenging task due to depth ambiguity and self-occlusion. We propose Multi-Hypothesis Transformer (MHFormer) that learns to represent multiple plausible pose hypotheses. MHFormer achieves state-of-the-art results on two challenging datasets: Human3.6M and MPI-INF-3DHP.
  arXiv  Detail & Related papers  (2021-11-24T18:59:02Z)
• Stochastic Modeling for Learnable Human Pose Triangulation [0.7646713951724009]
  We propose a modeling framework for 3D human pose triangulation and evaluate its performance across different datasets and spatial camera arrangements. The proposed pose triangulation model successfully generalizes to different camera arrangements and between two public datasets.
  arXiv  Detail & Related papers  (2021-10-01T09:26:25Z)
• Multi-person 3D Pose Estimation in Crowded Scenes Based on Multi-View Geometry [62.29762409558553]
  Epipolar constraints are at the core of feature matching and depth estimation in multi-person 3D human pose estimation methods. Despite the satisfactory performance of this formulation in sparser crowd scenes, its effectiveness is frequently challenged under denser crowd circumstances. In this paper, we depart from the multi-person 3D pose estimation formulation, and instead reformulate it as crowd pose estimation.
  arXiv  Detail & Related papers  (2020-07-21T17:59:36Z)
• Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation [58.72192168935338]
  Generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable. We propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our proposed model employs three consecutive differentiable transformations named as forward-kinematics, camera-projection and spatial-map transformation.
  arXiv  Detail & Related papers  (2020-06-24T23:56:33Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.