Convolutional Autoencoders for Human Motion Infilling

• URL: http://arxiv.org/abs/2010.11531v1
• Date: Thu, 22 Oct 2020 08:45:38 GMT
• Title: Convolutional Autoencoders for Human Motion Infilling
• Authors: Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, Otmar Hilliges
• Abstract summary: Motion infilling aims to complete the missing gap in between, such that the filled in poses plausibly forecast the start sequence and naturally transition into the end sequence. We show that a single model can be used to create natural transitions between different types of activities. Our method is not only able to fill in entire missing frames, but it can also be used to complete gaps where partial poses are available.
• Score: 37.16099544563645
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we propose a convolutional autoencoder to address the problem of motion infilling for 3D human motion data. Given a start and end sequence, motion infilling aims to complete the missing gap in between, such that the filled in poses plausibly forecast the start sequence and naturally transition into the end sequence. To this end, we propose a single, end-to-end trainable convolutional autoencoder. We show that a single model can be used to create natural transitions between different types of activities. Furthermore, our method is not only able to fill in entire missing frames, but it can also be used to complete gaps where partial poses are available (e.g. from end effectors), or to clean up other forms of noise (e.g. Gaussian). Also, the model can fill in an arbitrary number of gaps that potentially vary in length. In addition, no further post-processing on the model's outputs is necessary such as smoothing or closing discontinuities at the end of the gap. At the heart of our approach lies the idea to cast motion infilling as an inpainting problem and to train a convolutional de-noising autoencoder on image-like representations of motion sequences. At training time, blocks of columns are removed from such images and we ask the model to fill in the gaps. We demonstrate the versatility of the approach via a number of complex motion sequences and report on thorough evaluations performed to better understand the capabilities and limitations of the proposed approach.

Related papers

• Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer [62.29951737214263]
  Existing algorithms directly generate the full sequence which is expensive and prone to errors. We propose KeyMotion, that generates plausible human motion sequences corresponding to input text. We use a Variationalcoder (VAE) with Kullback-Leibler regularization to project the Autoencoder into a latent space. For the reverse diffusion, we propose a novel Parallel Skip Transformer that performs cross-modal attention between the design latents and text condition.
  arXiv  Detail & Related papers  (2024-05-24T11:12:37Z)
• RoHM: Robust Human Motion Reconstruction via Diffusion [58.63706638272891]
  RoHM is an approach for robust 3D human motion reconstruction from monocular RGB(-D) videos. It conditioned on noisy and occluded input data, reconstructs complete, plausible motions in consistent global coordinates. Our method outperforms state-of-the-art approaches qualitatively and quantitatively, while being faster at test time.
  arXiv  Detail & Related papers  (2024-01-16T18:57:50Z)
• Learning Snippet-to-Motion Progression for Skeleton-based Human Motion Prediction [14.988322340164391]
  Existing Graph Convolutional Networks to achieve human motion prediction largely adopt a one-step scheme. We observe that human motions have transitional patterns and can be split into snippets representative of each transition. We propose a snippet-to-motion multi-stage framework that breaks motion prediction into sub-tasks easier to accomplish.
  arXiv  Detail & Related papers  (2023-07-26T07:36:38Z)
• Shuffled Autoregression For Motion Interpolation [53.61556200049156]
  This work aims to provide a deep-learning solution for the motion task. We propose a novel framework, referred to as emphShuffled AutoRegression, which expands the autoregression to generate in arbitrary (shuffled) order. We also propose an approach to constructing a particular kind of dependency graph, with three stages assembled into an end-to-end spatial-temporal motion Transformer.
  arXiv  Detail & Related papers  (2023-06-10T07:14:59Z)
• HumanMAC: Masked Motion Completion for Human Motion Prediction [62.279925754717674]
  Human motion prediction is a classical problem in computer vision and computer graphics. Previous effects achieve great empirical performance based on an encoding-decoding style. In this paper, we propose a novel framework from a new perspective.
  arXiv  Detail & Related papers  (2023-02-07T18:34:59Z)
• Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance [83.25826307000717]
  We study the challenging problem of recovering detailed motion from a single motion-red image. Existing solutions to this problem estimate a single image sequence without considering the motion ambiguity for each region. In this paper, we explicitly account for such motion ambiguity, allowing us to generate multiple plausible solutions all in sharp detail.
  arXiv  Detail & Related papers  (2022-07-20T18:05:53Z)
• OTPose: Occlusion-Aware Transformer for Pose Estimation in Sparsely-Labeled Videos [21.893572076171527]
  We propose a method that leverages an attention mask for occluded joints and encodes temporal dependency between frames using transformers. We achieve state-of-the-art pose estimation results for PoseTrack 2017 and PoseTrack 2018 datasets.
  arXiv  Detail & Related papers  (2022-07-20T08:06:06Z)
• Single-Shot Motion Completion with Transformer [0.0]
  We propose a simple but effective method to solve multiple motion completion problems under a unified framework. Inspired by the recent great success of attention-based models, we consider the completion as a sequence to sequence prediction problem. Our method can run in a non-autoregressive manner and predict multiple missing frames within a single forward propagation in real time.
  arXiv  Detail & Related papers  (2021-03-01T06:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.