Morph: A Motion-free Physics Optimization Framework for Human Motion Generation

• URL: http://arxiv.org/abs/2411.14951v1
• Date: Fri, 22 Nov 2024 14:09:56 GMT
• Title: Morph: A Motion-free Physics Optimization Framework for Human Motion Generation
• Authors: Zhuo Li, Mingshuang Luo, Ruibing Hou, Xin Zhao, Hao Liu, Hong Chang, Zimo Liu, Chen Li,
• Abstract summary: Our framework achieves state-of-the-art motion generation quality while improving physical plausibility drastically. Experiments on text-to-motion and music-to-dance generation tasks demonstrate that our framework achieves state-of-the-art motion generation quality.
• Score: 25.51726849102517
• License:
• Abstract: Human motion generation plays a vital role in applications such as digital humans and humanoid robot control. However, most existing approaches disregard physics constraints, leading to the frequent production of physically implausible motions with pronounced artifacts such as floating and foot sliding. In this paper, we propose \textbf{Morph}, a \textbf{Mo}tion-f\textbf{r}ee \textbf{ph}ysics optimization framework, comprising a Motion Generator and a Motion Physics Refinement module, for enhancing physical plausibility without relying on costly real-world motion data. Specifically, the Motion Generator is responsible for providing large-scale synthetic motion data, while the Motion Physics Refinement Module utilizes these synthetic data to train a motion imitator within a physics simulator, enforcing physical constraints to project the noisy motions into a physically-plausible space. These physically refined motions, in turn, are used to fine-tune the Motion Generator, further enhancing its capability. Experiments on both text-to-motion and music-to-dance generation tasks demonstrate that our framework achieves state-of-the-art motion generation quality while improving physical plausibility drastically.

Related papers

• MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotations [85.85596165472663]
  We build MotionBank, which comprises 13 video action datasets, 1.24M motion sequences, and 132.9M frames of natural and diverse human motions. Our MotionBank is beneficial for general motion-related tasks of human motion generation, motion in-context generation, and motion understanding.
  arXiv  Detail & Related papers  (2024-10-17T17:31:24Z)
• Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes [83.55301458112672]
  Sitcom-Crafter is a system for human motion generation in 3D space. Central to the function generation modules is our novel 3D scene-aware human-human interaction module. Augmentation modules encompass plot comprehension for command generation, motion synchronization for seamless integration of different motion types.
  arXiv  Detail & Related papers  (2024-10-14T17:56:19Z)
• ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model [9.525806425270428]
  We present emphReinDiffuse that combines reinforcement learning with motion diffusion model to generate physically credible human motions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions, making them compatible with reinforcement learning paradigms. Our approach outperforms existing state-of-the-art models on two major datasets, HumanML3D and KIT-ML.
  arXiv  Detail & Related papers  (2024-10-09T16:24:11Z)
• DreamPhysics: Learning Physical Properties of Dynamic 3D Gaussians with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors. We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• Universal Humanoid Motion Representations for Physics-Based Control [71.46142106079292]
  We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. We first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator.
  arXiv  Detail & Related papers  (2023-10-06T20:48:43Z)
• DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics [21.00283279991885]
  We introduce DROP, a novel framework for modeling Dynamics Responses of humans using generative mOtion prior and Projective dynamics. We conduct extensive evaluations of our model across different motion tasks and various physical perturbations, demonstrating the scalability and diversity of responses.
  arXiv  Detail & Related papers  (2023-09-24T20:25:59Z)
• PhysDiff: Physics-Guided Human Motion Diffusion Model [101.1823574561535]
  Existing motion diffusion models largely disregard the laws of physics in the diffusion process. PhysDiff incorporates physical constraints into the diffusion process. Our approach achieves state-of-the-art motion quality and improves physical plausibility drastically.
  arXiv  Detail & Related papers  (2022-12-05T18:59:52Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)
• MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model [35.32967411186489]
  MotionDiffuse is a diffusion model-based text-driven motion generation framework. It excels at modeling complicated data distribution and generating vivid motion sequences. It responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts.
  arXiv  Detail & Related papers  (2022-08-31T17:58:54Z)
• Physics-based Human Motion Estimation and Synthesis from Videos [0.0]
  We propose a framework for training generative models of physically plausible human motion directly from monocular RGB videos. At the core of our method is a novel optimization formulation that corrects imperfect image-based pose estimations. Results show that our physically-corrected motions significantly outperform prior work on pose estimation.
  arXiv  Detail & Related papers  (2021-09-21T01:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.