3D Skeleton-based Human Motion Prediction with Manifold-Aware GAN
- URL: http://arxiv.org/abs/2203.00736v1
- Date: Tue, 1 Mar 2022 20:49:13 GMT
- Title: 3D Skeleton-based Human Motion Prediction with Manifold-Aware GAN
- Authors: Baptiste Chopin, Naima Otberdout, Mohamed Daoudi, Angela Bartolo
- Abstract summary: We propose a novel solution for 3D skeleton-based human motion prediction.
We build a manifold-aware Wasserstein generative adversarial model that captures the temporal and spatial dependencies of human motion.
Experiments have been conducted on CMU MoCap and Human 3.6M datasets.
- Score: 3.1313293632309827
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this work we propose a novel solution for 3D skeleton-based human motion
prediction. The objective of this task consists in forecasting future human
poses based on a prior skeleton pose sequence. This involves solving two main
challenges still present in recent literature; (1) discontinuity of the
predicted motion which results in unrealistic motions and (2) performance
deterioration in long-term horizons resulting from error accumulation across
time. We tackle these issues by using a compact manifold-valued representation
of 3D human skeleton motion. Specifically, we model the temporal evolution of
the 3D poses as trajectory, what allows us to map human motions to single
points on a sphere manifold. Using such a compact representation avoids error
accumulation and provides robust representation for long-term prediction while
ensuring the smoothness and the coherence of the whole motion. To learn these
non-Euclidean representations, we build a manifold-aware Wasserstein generative
adversarial model that captures the temporal and spatial dependencies of human
motion through different losses. Experiments have been conducted on CMU MoCap
and Human 3.6M datasets and demonstrate the superiority of our approach over
the state-of-the-art both in short and long term horizons. The smoothness of
the generated motion is highlighted in the qualitative results.
Related papers
- Hybrid 3D Human Pose Estimation with Monocular Video and Sparse IMUs [15.017274891943162]
Temporal 3D human pose estimation from monocular videos is a challenging task in human-centered computer vision.
Inertial sensor has been introduced to provide complementary source of information.
It remains challenging to integrate heterogeneous sensor data for producing physically rational 3D human poses.
arXiv Detail & Related papers (2024-04-27T09:02:42Z) - Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence [47.16903508897047]
In this study, we elucidate that variations in human appearance depend not only on the current frame's pose condition but also on past pose states.
We introduce Dyco, a novel method utilizing the delta pose sequence representation for non-rigid deformations.
In addition, our inertia-aware 3D human method can unprecedentedly simulate appearance changes caused by inertia at different velocities.
arXiv Detail & Related papers (2024-03-28T06:05:14Z) - Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion
Modeling [83.76377808476039]
We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior.
Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton.
A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
arXiv Detail & Related papers (2023-08-18T16:41:57Z) - Investigating Pose Representations and Motion Contexts Modeling for 3D
Motion Prediction [63.62263239934777]
We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task.
We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction.
Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
arXiv Detail & Related papers (2021-12-30T10:45:22Z) - Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
We introduce a unified deep generative network for both diverse and controllable motion prediction.
Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
arXiv Detail & Related papers (2021-08-19T00:58:00Z) - Multi-frame sequence generator of 4D human body motion [0.0]
We propose a generative auto-encoder-based framework, which encodes, global locomotion including translation and rotation, and multi-frame temporal motion as a single latent space vector.
Our results validate the ability of the model to reconstruct 4D sequences of human morphology within a low error bound.
We also illustrate the benefits of the approach for 4D human motion prediction of future frames from initial human frames.
arXiv Detail & Related papers (2021-06-07T13:56:46Z) - Human Motion Prediction Using Manifold-Aware Wasserstein GAN [4.771549505875783]
We build a manifold-aware Wasserstein generative adversarial model that captures the temporal and spatial dependencies of human motion.
Our approach outperforms the state-of-the-art on CMU MoCap and Human 3.6M datasets.
arXiv Detail & Related papers (2021-05-18T17:56:10Z) - We are More than Our Joints: Predicting how 3D Bodies Move [63.34072043909123]
We train a novel variational autoencoder that generates motions from latent frequencies.
Experiments show that our method produces state-of-the-art results and realistic 3D body animations.
arXiv Detail & Related papers (2020-12-01T16:41:04Z) - Perpetual Motion: Generating Unbounded Human Motion [61.40259979876424]
We focus on long-term prediction; that is, generating long sequences of human motion that is plausible.
We propose a model to generate non-deterministic, textitever-changing, perpetual human motion.
We train this using a heavy-tailed function of the KL divergence of a white-noise Gaussian process, allowing latent sequence temporal dependency.
arXiv Detail & Related papers (2020-07-27T21:50:36Z) - Motion Guided 3D Pose Estimation from Videos [81.14443206968444]
We propose a new loss function, called motion loss, for the problem of monocular 3D Human pose estimation from 2D pose.
In computing motion loss, a simple yet effective representation for keypoint motion, called pairwise motion encoding, is introduced.
We design a new graph convolutional network architecture, U-shaped GCN (UGCN), which captures both short-term and long-term motion information.
arXiv Detail & Related papers (2020-04-29T06:59:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.