Exploring Versatile Prior for Human Motion via Motion Frequency Guidance

• URL: http://arxiv.org/abs/2111.13074v1
• Date: Thu, 25 Nov 2021 13:24:44 GMT
• Title: Exploring Versatile Prior for Human Motion via Motion Frequency Guidance
• Authors: Jiachen Xu, Min Wang, Jingyu Gong, Wentao Liu, Chen Qian, Yuan Xie, Lizhuang Ma
• Abstract summary: We learn a framework to learn the versatile motion prior, which models the inherent probability distribution of human motions. For efficient prior representation learning, we propose a global orientation normalization to remove redundant environment information. We then adopt a denoising training scheme to disentangle the environment information from input motion data in a learnable way.
• Score: 32.50770614788775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior plays an important role in providing the plausible constraint on human motion. Previous works design motion priors following a variety of paradigms under different circumstances, leading to the lack of versatility. In this paper, we first summarize the indispensable properties of the motion prior, and accordingly, design a framework to learn the versatile motion prior, which models the inherent probability distribution of human motions. Specifically, for efficient prior representation learning, we propose a global orientation normalization to remove redundant environment information in the original motion data space. Also, a two-level, sequence-based and segment-based, frequency guidance is introduced into the encoding stage. Then, we adopt a denoising training scheme to disentangle the environment information from input motion data in a learnable way, so as to generate consistent and distinguishable representation. Embedding our motion prior into prevailing backbones on three different tasks, we conduct extensive experiments, and both quantitative and qualitative results demonstrate the versatility and effectiveness of our motion prior. Our model and code are available at https://github.com/JchenXu/human-motion-prior.

Related papers

• Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction [7.150292351809277]
  Action-driven human motion prediction aims to forecast future human motion based on the observed sequence. It requires modeling the smooth yet realistic transition between multiple action labels. We generalize our trained in-betweening learning model on one dataset to two unseen large-scale motion datasets to produce natural transitions.
  arXiv  Detail & Related papers  (2023-10-23T13:16:51Z)
• Universal Humanoid Motion Representations for Physics-Based Control [71.46142106079292]
  We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. We first learn a motion imitator that can imitate all of human motion from a large, unstructured motion dataset. We then create our motion representation by distilling skills directly from the imitator.
  arXiv  Detail & Related papers  (2023-10-06T20:48:43Z)
• DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion [70.33381660741861]
  We present DiverseMotion, a new approach for synthesizing high-quality human motions conditioned on textual descriptions. We show that our DiverseMotion achieves the state-of-the-art motion quality and competitive motion diversity.
  arXiv  Detail & Related papers  (2023-09-04T05:43:48Z)
• Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
  We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation. M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
  arXiv  Detail & Related papers  (2023-08-28T10:40:16Z)
• Neural Categorical Priors for Physics-Based Character Control [12.731392285646614]
  We propose a new learning framework for controlling physics-based characters with significantly improved motion quality and diversity. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game.
  arXiv  Detail & Related papers  (2023-08-14T15:10:29Z)
• TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction [1.8923948104852863]
  We propose TransFusion, an innovative and practical diffusion-based model for 3D human motion prediction. Our model leverages Transformer as the backbone with long skip connections between shallow and deep layers. In contrast to prior diffusion-based models that utilize extra modules like cross-attention and adaptive layer normalization, we treat all inputs, including conditions, as tokens to create a more lightweight model.
  arXiv  Detail & Related papers  (2023-07-30T01:52:07Z)
• Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Video [16.32910684198013]
  We present a novel multi-frame human pose estimation framework, which employs temporal differences across frames to model dynamic contexts. To be specific, we design a multi-stage entangled learning sequences conditioned on multi-stage differences to derive informative motion representation sequences. These place us to rank No.1 in the Crowd Pose Estimation in Complex Events Challenge on benchmark HiEve.
  arXiv  Detail & Related papers  (2023-03-15T09:29:03Z)
• HumanMAC: Masked Motion Completion for Human Motion Prediction [62.279925754717674]
  Human motion prediction is a classical problem in computer vision and computer graphics. Previous effects achieve great empirical performance based on an encoding-decoding style. In this paper, we propose a novel framework from a new perspective.
  arXiv  Detail & Related papers  (2023-02-07T18:34:59Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.