HumanMAC: Masked Motion Completion for Human Motion Prediction

• URL: http://arxiv.org/abs/2302.03665v4
• Date: Mon, 14 Aug 2023 12:31:19 GMT
• Title: HumanMAC: Masked Motion Completion for Human Motion Prediction
• Authors: Ling-Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, Tongliang Liu
• Abstract summary: Human motion prediction is a classical problem in computer vision and computer graphics. Previous effects achieve great empirical performance based on an encoding-decoding style. In this paper, we propose a novel framework from a new perspective.
• Score: 62.279925754717674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human motion prediction is a classical problem in computer vision and computer graphics, which has a wide range of practical applications. Previous effects achieve great empirical performance based on an encoding-decoding style. The methods of this style work by first encoding previous motions to latent representations and then decoding the latent representations into predicted motions. However, in practice, they are still unsatisfactory due to several issues, including complicated loss constraints, cumbersome training processes, and scarce switch of different categories of motions in prediction. In this paper, to address the above issues, we jump out of the foregoing style and propose a novel framework from a new perspective. Specifically, our framework works in a masked completion fashion. In the training stage, we learn a motion diffusion model that generates motions from random noise. In the inference stage, with a denoising procedure, we make motion prediction conditioning on observed motions to output more continuous and controllable predictions. The proposed framework enjoys promising algorithmic properties, which only needs one loss in optimization and is trained in an end-to-end manner. Additionally, it accomplishes the switch of different categories of motions effectively, which is significant in realistic tasks, e.g., the animation task. Comprehensive experiments on benchmarks confirm the superiority of the proposed framework. The project page is available at https://lhchen.top/Human-MAC.

Related papers

• Existence Is Chaos: Enhancing 3D Human Motion Prediction with Uncertainty Consideration [27.28184416632815]
  We argue that the recorded motion in training data could be an observation of possible future, rather than a predetermined result. A novel computationally efficient encoder-decoder model with uncertainty consideration is proposed.
  arXiv  Detail & Related papers  (2024-03-21T03:34:18Z)
• Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
  We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications. Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
  arXiv  Detail & Related papers  (2023-12-14T12:57:35Z)
• Neural Categorical Priors for Physics-Based Character Control [12.731392285646614]
  We propose a new learning framework for controlling physics-based characters with significantly improved motion quality and diversity. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game.
  arXiv  Detail & Related papers  (2023-08-14T15:10:29Z)
• PREF: Predictability Regularized Neural Motion Fields [68.60019434498703]
  Knowing 3D motions in a dynamic scene is essential to many vision applications. We leverage a neural motion field for estimating the motion of all points in a multiview setting. We propose to regularize the estimated motion to be predictable.
  arXiv  Detail & Related papers  (2022-09-21T22:32:37Z)
• Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction [81.94175022575966]
  We introduce the task of action-driven human motion prediction. It aims to predict multiple plausible future motions given a sequence of action labels and a short motion history.
  arXiv  Detail & Related papers  (2022-05-31T08:38:07Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Exploring Versatile Prior for Human Motion via Motion Frequency Guidance [32.50770614788775]
  We learn a framework to learn the versatile motion prior, which models the inherent probability distribution of human motions. For efficient prior representation learning, we propose a global orientation normalization to remove redundant environment information. We then adopt a denoising training scheme to disentangle the environment information from input motion data in a learnable way.
  arXiv  Detail & Related papers  (2021-11-25T13:24:44Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)
• Single-Shot Motion Completion with Transformer [0.0]
  We propose a simple but effective method to solve multiple motion completion problems under a unified framework. Inspired by the recent great success of attention-based models, we consider the completion as a sequence to sequence prediction problem. Our method can run in a non-autoregressive manner and predict multiple missing frames within a single forward propagation in real time.
  arXiv  Detail & Related papers  (2021-03-01T06:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.