Human Motion Diffusion Model
- URL: http://arxiv.org/abs/2209.14916v2
- Date: Mon, 3 Oct 2022 09:17:41 GMT
- Title: Human Motion Diffusion Model
- Authors: Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or
and Amit H. Bermano
- Abstract summary: Motion Diffusion Model (MDM) is a transformer-based generative model for the human motion domain.
We show that our model is trained with lightweight resources and yet achieves state-of-the-art results on leading benchmarks for text-to-motion and action-to-motion.
- Score: 35.05219668478535
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Natural and expressive human motion generation is the holy grail of computer
animation. It is a challenging task, due to the diversity of possible motion,
human perceptual sensitivity to it, and the difficulty of accurately describing
it. Therefore, current generative solutions are either low-quality or limited
in expressiveness. Diffusion models, which have already shown remarkable
generative capabilities in other domains, are promising candidates for human
motion due to their many-to-many nature, but they tend to be resource hungry
and hard to control. In this paper, we introduce Motion Diffusion Model (MDM),
a carefully adapted classifier-free diffusion-based generative model for the
human motion domain. MDM is transformer-based, combining insights from motion
generation literature. A notable design-choice is the prediction of the sample,
rather than the noise, in each diffusion step. This facilitates the use of
established geometric losses on the locations and velocities of the motion,
such as the foot contact loss. As we demonstrate, MDM is a generic approach,
enabling different modes of conditioning, and different generation tasks. We
show that our model is trained with lightweight resources and yet achieves
state-of-the-art results on leading benchmarks for text-to-motion and
action-to-motion. https://guytevet.github.io/mdm-page/ .
Related papers
- Text-driven Human Motion Generation with Motion Masked Diffusion Model [23.637853270123045]
Text human motion generation is a task that synthesizes human motion sequences conditioned on natural language.
Current diffusion model-based approaches have outstanding performance in the diversity and multimodality of generation.
We propose Motion Masked Diffusion Model bftext(MMDM), a novel human motion mechanism for diffusion model.
arXiv Detail & Related papers (2024-09-29T12:26:24Z) - HUMOS: Human Motion Model Conditioned on Body Shape [54.20419874234214]
We introduce a new approach to develop a generative motion model based on body shape.
We show that it's possible to train this model using unpaired data.
The resulting model generates diverse, physically plausible, and dynamically stable human motions.
arXiv Detail & Related papers (2024-09-05T23:50:57Z) - Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications.
Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
arXiv Detail & Related papers (2023-12-14T12:57:35Z) - Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation.
M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse.
We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
arXiv Detail & Related papers (2023-08-28T10:40:16Z) - Single Motion Diffusion [33.81898532874481]
We present SinMDM, a model designed to learn the internal motifs of a single motion sequence with arbitrary topology and synthesize motions of arbitrary length that are faithful to them.
SinMDM can be applied in various contexts, including spatial and temporal in-betweening, motion expansion, style transfer, and crowd animation.
Our results show that SinMDM outperforms existing methods both in quality and time-space efficiency.
arXiv Detail & Related papers (2023-02-12T13:02:19Z) - Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs.
Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
arXiv Detail & Related papers (2022-12-08T03:07:00Z) - MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model [35.32967411186489]
MotionDiffuse is a diffusion model-based text-driven motion generation framework.
It excels at modeling complicated data distribution and generating vivid motion sequences.
It responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts.
arXiv Detail & Related papers (2022-08-31T17:58:54Z) - Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
We introduce a unified deep generative network for both diverse and controllable motion prediction.
Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
arXiv Detail & Related papers (2021-08-19T00:58:00Z) - Perpetual Motion: Generating Unbounded Human Motion [61.40259979876424]
We focus on long-term prediction; that is, generating long sequences of human motion that is plausible.
We propose a model to generate non-deterministic, textitever-changing, perpetual human motion.
We train this using a heavy-tailed function of the KL divergence of a white-noise Gaussian process, allowing latent sequence temporal dependency.
arXiv Detail & Related papers (2020-07-27T21:50:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.