Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction

• URL: http://arxiv.org/abs/2501.06035v1
• Date: Fri, 10 Jan 2025 15:13:43 GMT
• Title: Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction
• Authors: Cecilia Curreli, Dominik Muhle, Abhishek Saroha, Zhenzhang Ye, Riccardo Marin, Daniel Cremers,
• Abstract summary: SkeletonDiffusion is a latent diffusion model that embeds an explicit inductive bias on the human body. It consistently generates realistic predictions while avoiding artifacts such as limb distortion. It sets a new benchmark on three real-world datasets.
• Score: 36.0214360692653
• License:
• Abstract: Probabilistic human motion prediction aims to forecast multiple possible future movements from past observations. While current approaches report high diversity and realism, they often generate motions with undetected limb stretching and jitter. To address this, we introduce SkeletonDiffusion, a latent diffusion model that embeds an explicit inductive bias on the human body within its architecture and training. Our model is trained with a novel nonisotropic Gaussian diffusion formulation that aligns with the natural kinematic structure of the human skeleton. Results show that our approach outperforms conventional isotropic alternatives, consistently generating realistic predictions while avoiding artifacts such as limb distortion. Additionally, we identify a limitation in commonly used diversity metrics, which may inadvertently favor models that produce inconsistent limb lengths within the same sequence. SkeletonDiffusion sets a new benchmark on three real-world datasets, outperforming various baselines across multiple evaluation metrics. Visit our project page: https://ceveloper.github.io/publications/skeletondiffusion/

Related papers

• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction [1.8923948104852863]
  We propose TransFusion, an innovative and practical diffusion-based model for 3D human motion prediction. Our model leverages Transformer as the backbone with long skip connections between shallow and deep layers. In contrast to prior diffusion-based models that utilize extra modules like cross-attention and adaptive layer normalization, we treat all inputs, including conditions, as tokens to create a more lightweight model.
  arXiv  Detail & Related papers  (2023-07-30T01:52:07Z)
• Uncertainty-Aware Pedestrian Trajectory Prediction via Distributional Diffusion [26.715578412088327]
  We present a model-agnostic uncertainty-aware pedestrian trajectory prediction framework. Unlike previous studies, we translate the predictiveity to explicit distributions, allowing it to generate plausible future trajectories. Our framework is compatible with different neural net architectures.
  arXiv  Detail & Related papers  (2023-03-15T04:58:43Z)
• Can We Use Diffusion Probabilistic Models for 3D Motion Prediction? [10.808563617061846]
  This paper presents a study of employing diffusion probabilistic models to predict future 3D human motion from the previously observed motion. Based on the Human 3.6M and HumanEva-I datasets, our results show that diffusion probabilistic models are competitive for both single (deterministic) and multiple (stochastic) 3D motion prediction tasks.
  arXiv  Detail & Related papers  (2023-02-28T11:34:55Z)
• BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction [26.306489700180627]
  We present BeLFusion, a model that leverages latent diffusion models in human motion prediction (HMP) to sample from a latent space where behavior is disentangled from pose and motion. Thanks to our behavior coupler's ability to transfer sampled behavior to ongoing motion, BeLFusion's predictions display a variety of behaviors that are significantly more realistic than the state of the art.
  arXiv  Detail & Related papers  (2022-11-25T18:59:03Z)
• Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Prediction [22.354538952573158]
  MotionDiff is a diffusion probabilistic model to treat the kinematics of human joints as heated particles. MotionDiff consists of two parts: a spatial-temporal transformer-based diffusion network to generate diverse yet plausible motions, and a graph convolutional network to further refine the outputs.
  arXiv  Detail & Related papers  (2022-10-12T07:38:33Z)
• A generic diffusion-based approach for 3D human pose prediction in the wild [68.00961210467479]
  3D human pose forecasting, i.e., predicting a sequence of future human 3D poses given a sequence of past observed ones, is a challenging-temporal task. We provide a unified formulation in which incomplete elements (no matter in the prediction or observation) are treated as noise and propose a conditional diffusion model that denoises them and forecasts plausible poses. We investigate our findings on four standard datasets and obtain significant improvements over the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-11T17:59:54Z)
• Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion [88.45326906116165]
  We present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID) We encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories. Experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-03-25T16:59:08Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.