DivDiff: A Conditional Diffusion Model for Diverse Human Motion Prediction

• URL: http://arxiv.org/abs/2409.00014v1
• Date: Fri, 16 Aug 2024 04:51:32 GMT
• Title: DivDiff: A Conditional Diffusion Model for Diverse Human Motion Prediction
• Authors: Hua Yu, Yaqing Hou, Wenbin Pei, Qiang Zhang,
• Abstract summary: We propose a conditional diffusion-based generative model, called DivDiff, to predict more diverse and realistic human motions. Specifically, the DivDiff employs DDPM as our backbone and incorporates Discrete Cosine Transform (DCT) and transformer mechanisms. We design a diversified reinforcement sampling function (DRSF) to enforce human skeletal constraints on the predicted human motions.
• Score: 9.447439259813112
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diverse human motion prediction (HMP) aims to predict multiple plausible future motions given an observed human motion sequence. It is a challenging task due to the diversity of potential human motions while ensuring an accurate description of future human motions. Current solutions are either low-diversity or limited in expressiveness. Recent denoising diffusion models (DDPM) hold potential generative capabilities in generative tasks. However, introducing DDPM directly into diverse HMP incurs some issues. Although DDPM can increase the diversity of the potential patterns of human motions, the predicted human motions become implausible over time because of the significant noise disturbances in the forward process of DDPM. This phenomenon leads to the predicted human motions being hard to control, seriously impacting the quality of predicted motions and restricting their practical applicability in real-world scenarios. To alleviate this, we propose a novel conditional diffusion-based generative model, called DivDiff, to predict more diverse and realistic human motions. Specifically, the DivDiff employs DDPM as our backbone and incorporates Discrete Cosine Transform (DCT) and transformer mechanisms to encode the observed human motion sequence as a condition to instruct the reverse process of DDPM. More importantly, we design a diversified reinforcement sampling function (DRSF) to enforce human skeletal constraints on the predicted human motions. DRSF utilizes the acquired information from human skeletal as prior knowledge, thereby reducing significant disturbances introduced during the forward process. Extensive results received in the experiments on two widely-used datasets (Human3.6M and HumanEva-I) demonstrate that our model obtains competitive performance on both diversity and accuracy.

Related papers

• MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds [20.83684434910106]
  We present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Specifically, we propose novel decoupled joint acceleration to model human dynamics from existing limited motion data. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks.
  arXiv  Detail & Related papers  (2024-09-01T15:00:16Z)
• COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation [98.05046790227561]
  COIN is a control-inpainting motion diffusion prior that enables fine-grained control to disentangle human and camera motions. COIN outperforms the state-of-the-art methods in terms of global human motion estimation and camera motion estimation.
  arXiv  Detail & Related papers  (2024-08-29T10:36:29Z)
• Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction [46.309401205546656]
  Real-world human movements are goal-directed and highly influenced by the spatial layout of their surrounding scenes. We propose a Multi-Condition Latent Diffusion network (MCLD) that reformulates the human motion prediction task as a multi-condition joint inference problem. Our network achieves significant improvements over the state-of-the-art methods on both realistic and diverse predictions.
  arXiv  Detail & Related papers  (2024-05-29T02:21:31Z)
• AdvMT: Adversarial Motion Transformer for Long-term Human Motion Prediction [2.837740438355204]
  We present the Adversarial Motion Transformer (AdvMT), a novel model that integrates a transformer-based motion encoder and a temporal continuity discriminator. With adversarial training, our method effectively reduces the unwanted artifacts in predictions, thereby ensuring the learning of more realistic and fluid human motions.
  arXiv  Detail & Related papers  (2024-01-10T09:15:50Z)
• TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction [1.8923948104852863]
  We propose TransFusion, an innovative and practical diffusion-based model for 3D human motion prediction. Our model leverages Transformer as the backbone with long skip connections between shallow and deep layers. In contrast to prior diffusion-based models that utilize extra modules like cross-attention and adaptive layer normalization, we treat all inputs, including conditions, as tokens to create a more lightweight model.
  arXiv  Detail & Related papers  (2023-07-30T01:52:07Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)
• A Neural Active Inference Model of Perceptual-Motor Learning [62.39667564455059]
  The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans. We present a novel formulation of the prior function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy.
  arXiv  Detail & Related papers  (2022-11-16T20:00:38Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks [9.06677862854201]
  We propose a novel approach to predict future human motions from a single image, with mixture density networks (MDN) modeling. Contrary to most existing deep human motion prediction approaches, the multimodal nature of MDN enables the generation of diverse future motion hypotheses. Our trained model directly takes an image as input and generates multiple plausible motions that satisfy the given condition.
  arXiv  Detail & Related papers  (2021-09-13T08:49:33Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.