A generic diffusion-based approach for 3D human pose prediction in the wild

• URL: http://arxiv.org/abs/2210.05669v1
• Date: Tue, 11 Oct 2022 17:59:54 GMT
• Title: A generic diffusion-based approach for 3D human pose prediction in the wild
• Authors: Saeed Saadatnejad, Ali Rasekh, Mohammadreza Mofayezi, Yasamin Medghalchi, Sara Rajabzadeh, Taylor Mordan, Alexandre Alahi
• Abstract summary: 3D human pose forecasting, i.e., predicting a sequence of future human 3D poses given a sequence of past observed ones, is a challenging-temporal task. We provide a unified formulation in which incomplete elements (no matter in the prediction or observation) are treated as noise and propose a conditional diffusion model that denoises them and forecasts plausible poses. We investigate our findings on four standard datasets and obtain significant improvements over the state-of-the-art.
• Score: 68.00961210467479
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D human pose forecasting, i.e., predicting a sequence of future human 3D poses given a sequence of past observed ones, is a challenging spatio-temporal task. It can be more challenging in real-world applications where occlusions will inevitably happen, and estimated 3D coordinates of joints would contain some noise. We provide a unified formulation in which incomplete elements (no matter in the prediction or observation) are treated as noise and propose a conditional diffusion model that denoises them and forecasts plausible poses. Instead of naively predicting all future frames at once, our model consists of two cascaded sub-models, each specialized for modeling short and long horizon distributions. We also propose a generic framework to improve any 3D pose forecasting model by leveraging our diffusion model in two additional steps: a pre-processing step to repair the inputs and a post-processing step to refine the outputs. We investigate our findings on four standard datasets (Human3.6M, HumanEva-I, AMASS, and 3DPW) and obtain significant improvements over the state-of-the-art. The code will be made available online.

Related papers

• UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
  UPose3D is a novel approach for multi-view 3D human pose estimation. It improves robustness and flexibility without requiring direct 3D annotations.
  arXiv  Detail & Related papers  (2024-04-23T00:18:00Z)
• D3PRefiner: A Diffusion-based Denoise Method for 3D Human Pose Refinement [3.514184876338779]
  A Diffusion-based 3D Pose Refiner is proposed to refine the output of any existing 3D pose estimator. We leverage the architecture of current diffusion models to convert the distribution of noisy 3D poses into ground truth 3D poses. Experimental results demonstrate the proposed architecture can significantly improve the performance of current sequence-to-sequence 3D pose estimators.
  arXiv  Detail & Related papers  (2024-01-08T14:21:02Z)
• DiffHPE: Robust, Coherent 3D Human Pose Lifting with Diffusion [54.0238087499699]
  We show that diffusion models enhance the accuracy, robustness, and coherence of human pose estimations. We introduce DiffHPE, a novel strategy for harnessing diffusion models in 3D-HPE. Our findings indicate that while standalone diffusion models provide commendable performance, their accuracy is even better in combination with supervised models.
  arXiv  Detail & Related papers  (2023-09-04T12:54:10Z)
• FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations [26.693664045454526]
  We present a generative approach to forecast long-term future human behavior in 3D, requiring only weak supervision from readily available 2D human action data. We jointly predict high-level coarse action labels together with their low-level fine-grained realizations as characteristic 3D human poses. Our experiments demonstrate the complementary nature of joint action and 3D pose prediction.
  arXiv  Detail & Related papers  (2022-11-25T18:59:53Z)
• Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction [63.3021778885906]
  3D bounding boxes are a widespread intermediate representation in many computer vision applications. We propose methods for leveraging our autoregressive model to make high confidence predictions and meaningful uncertainty measures. We release a simulated dataset, COB-3D, which highlights new types of ambiguity that arise in real-world robotics applications.
  arXiv  Detail & Related papers  (2022-10-13T23:57:40Z)
• Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimation [70.32536356351706]
  We introduce MRP-Net that constitutes a common deep network backbone with two output heads subscribing to two diverse configurations. We derive suitable measures to quantify prediction uncertainty at both pose and joint level. We present a comprehensive evaluation of the proposed approach and demonstrate state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T07:14:58Z)
• Probabilistic Modeling for Human Mesh Recovery [73.11532990173441]
  This paper focuses on the problem of 3D human reconstruction from 2D evidence. We recast the problem as learning a mapping from the input to a distribution of plausible 3D poses.
  arXiv  Detail & Related papers  (2021-08-26T17:55:11Z)
• Forecasting Characteristic 3D Poses of Human Actions [24.186058965796157]
  We propose the task of forecasting characteristic 3D poses from a monocular video observation of a person to predict a future 3D pose of that person in a likely action-defining, characteristic pose. We define a semantically meaningful pose prediction task that decouples the predicted pose from time, taking inspiration from goal-directed behavior. Our experiments with this dataset suggest that our proposed probabilistic approach outperforms state-of-the-art methods by 22% on average.
  arXiv  Detail & Related papers  (2020-11-30T18:20:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.