TrajeVAE -- Controllable Human Motion Generation from Trajectories
- URL: http://arxiv.org/abs/2104.00351v1
- Date: Thu, 1 Apr 2021 09:12:48 GMT
- Title: TrajeVAE -- Controllable Human Motion Generation from Trajectories
- Authors: Kacper Kania, Marek Kowalski, Tomasz Trzci\'nski
- Abstract summary: We propose a novel transformer-like architecture, TrajeVAE, that provides a versatile framework for 3D human animation.
We show that TrajeVAE outperforms trajectory-based reference approaches and methods that base their predictions on past poses in terms of accuracy.
- Score: 20.531400859656042
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: The generation of plausible and controllable 3D human motion animations is a
long-standing problem that often requires a manual intervention of skilled
artists. Existing machine learning approaches try to semi-automate this process
by allowing the user to input partial information about the future movement.
However, they are limited in two significant ways: they either base their pose
prediction on past prior frames with no additional control over the future
poses or allow the user to input only a single trajectory that precludes
fine-grained control over the output. To mitigate these two issues, we
reformulate the problem of future pose prediction into pose completion in space
and time where trajectories are represented as poses with missing joints. We
show that such a framework can generalize to other neural networks designed for
future pose prediction. Once trained in this framework, a model is capable of
predicting sequences from any number of trajectories. To leverage this notion,
we propose a novel transformer-like architecture, TrajeVAE, that provides a
versatile framework for 3D human animation. We demonstrate that TrajeVAE
outperforms trajectory-based reference approaches and methods that base their
predictions on past poses in terms of accuracy. We also show that it can
predict reasonable future poses even if provided only with an initial pose.
Related papers
- Past Movements-Guided Motion Representation Learning for Human Motion Prediction [0.0]
We propose a self-supervised learning framework designed to enhance motion representation.
The framework consists of two stages: first, the network is pretrained through the self-reconstruction of past sequences, and the guided reconstruction of future sequences based on past movements.
Our method reduces the average prediction errors by 8.8% across Human3.6, 3DPW, and AMASS datasets.
arXiv Detail & Related papers (2024-08-04T17:00:37Z) - Humanoid Locomotion as Next Token Prediction [84.21335675130021]
Our model is a causal transformer trained via autoregressive prediction of sensorimotor trajectories.
We show that our model enables a full-sized humanoid to walk in San Francisco zero-shot.
Our model can transfer to the real world even when trained on only 27 hours of walking data, and can generalize commands not seen during training like walking backward.
arXiv Detail & Related papers (2024-02-29T18:57:37Z) - Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior.
Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
arXiv Detail & Related papers (2023-12-26T18:56:49Z) - Learning Snippet-to-Motion Progression for Skeleton-based Human Motion
Prediction [14.988322340164391]
Existing Graph Convolutional Networks to achieve human motion prediction largely adopt a one-step scheme.
We observe that human motions have transitional patterns and can be split into snippets representative of each transition.
We propose a snippet-to-motion multi-stage framework that breaks motion prediction into sub-tasks easier to accomplish.
arXiv Detail & Related papers (2023-07-26T07:36:38Z) - A generic diffusion-based approach for 3D human pose prediction in the
wild [68.00961210467479]
3D human pose forecasting, i.e., predicting a sequence of future human 3D poses given a sequence of past observed ones, is a challenging-temporal task.
We provide a unified formulation in which incomplete elements (no matter in the prediction or observation) are treated as noise and propose a conditional diffusion model that denoises them and forecasts plausible poses.
We investigate our findings on four standard datasets and obtain significant improvements over the state-of-the-art.
arXiv Detail & Related papers (2022-10-11T17:59:54Z) - DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using
Attention-Based Generative Adverserial Network [9.247294820004143]
We propose a transformer-based generative model for forecasting multiple diverse human motions.
Our model first predicts the pose of the body relative to the hip joint. Then the textitHip Prediction Module predicts the trajectory of the hip movement for each predicted pose frame.
We show that our system outperforms the state-of-the-art in human motion prediction while it can predict diverse multi-motion future trajectories with hip movements.
arXiv Detail & Related papers (2022-09-13T23:22:33Z) - Investigating Pose Representations and Motion Contexts Modeling for 3D
Motion Prediction [63.62263239934777]
We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task.
We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction.
Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
arXiv Detail & Related papers (2021-12-30T10:45:22Z) - Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
We introduce a unified deep generative network for both diverse and controllable motion prediction.
Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
arXiv Detail & Related papers (2021-08-19T00:58:00Z) - Future Frame Prediction for Robot-assisted Surgery [57.18185972461453]
We propose a ternary prior guided variational autoencoder (TPG-VAE) model for future frame prediction in robotic surgical video sequences.
Besides content distribution, our model learns motion distribution, which is novel to handle the small movements of surgical tools.
arXiv Detail & Related papers (2021-03-18T15:12:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.