Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs

• URL: http://arxiv.org/abs/2109.10257v1
• Date: Tue, 21 Sep 2021 15:33:40 GMT
• Title: Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs
• Authors: Abduallah Mohamed, Huancheng Chen, Zhangyang Wang and Christian Claudel
• Abstract summary: We propose a deeptemporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones. By the design, Skeleton-Graph predicts the future 3D poses without divergence on the long-term unlike prior works. Our results show an FDE improvement of at least 27% and an ADE of 4% on both the GTA-IM and PROX datasets respectively.
• Score: 67.29552662707516
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Several applications such as autonomous driving, augmented reality and virtual reality requires a precise prediction of the 3D human pose. Recently, a new problem was introduced in the field to predict the 3D human poses from an observed 2D poses. We propose Skeleton-Graph, a deep spatio-temporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones. Unlike prior works, Skeleton-Graph focuses on modeling the interaction between the skeleton joints by exploiting their spatial configuration. This is being achieved by formulating the problem as a graph structure while learning a suitable graph adjacency kernel. By the design, Skeleton-Graph predicts the future 3D poses without divergence on the long-term unlike prior works. We also introduce a new metric that measures the divergence of predictions on the long-term. Our results show an FDE improvement of at least 27% and an ADE of 4% on both the GTA-IM and PROX datasets respectively in comparison with prior works. Also, we are 88% and 93% less divergence on the long-term motion prediction in comparison with prior works on both GTA-IM and PROX datasets. https://github.com/abduallahmohamed/Skeleton-Graph.git

Related papers

• Improving 3D Pose Estimation for Sign Language [38.20064386142944]
  This work addresses 3D human pose reconstruction in single images. We present a method that combines Forward Kinematics (FK) with neural networks to ensure a fast and valid prediction of 3D pose.
  arXiv  Detail & Related papers  (2023-08-18T13:05:10Z)
• Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction [120.08257447708503]
  Graph convolutional network based methods that model the body-joints' relations, have recently shown great promise in 3D skeleton-based human motion prediction. We propose a novel skeleton-parted graph scattering network (SPGSN) SPGSN outperforms state-of-the-art methods by remarkable margins of 13.8%, 9.3% and 2.7% in terms of 3D mean per joint position error (MPJPE) on Human3.6M, CMU Mocap and 3DPW datasets, respectively.
  arXiv  Detail & Related papers  (2022-07-31T05:51:39Z)
• 3D Human Pose Regression using Graph Convolutional Network [68.8204255655161]
  We propose a graph convolutional network named PoseGraphNet for 3D human pose regression from 2D poses. Our model's performance is close to the state-of-the-art, but with much fewer parameters.
  arXiv  Detail & Related papers  (2021-05-21T14:41:31Z)
• We are More than Our Joints: Predicting how 3D Bodies Move [63.34072043909123]
  We train a novel variational autoencoder that generates motions from latent frequencies. Experiments show that our method produces state-of-the-art results and realistic 3D body animations.
  arXiv  Detail & Related papers  (2020-12-01T16:41:04Z)
• HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation [7.559220068352681]
  We propose a lightweight model called HOPE-Net which jointly estimates hand and object pose in 2D and 3D in real-time. Our network uses a cascade of two adaptive graph convolutional neural networks, one to estimate 2D coordinates of the hand joints and object corners, followed by another to convert 2D coordinates to 3D.
  arXiv  Detail & Related papers  (2020-03-31T19:01:42Z)
• Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction [102.9787019197379]
  We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The model is action-category-agnostic and follows an encoder-decoder framework. The proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions.
  arXiv  Detail & Related papers  (2020-03-17T02:49:51Z)
• Anatomy-aware 3D Human Pose Estimation with Bone-based Pose Decomposition [92.99291528676021]
  Instead of directly regressing the 3D joint locations, we decompose the task into bone direction prediction and bone length prediction. Our motivation is the fact that the bone lengths of a human skeleton remain consistent across time. Our full model outperforms the previous best results on Human3.6M and MPI-INF-3DHP datasets.
  arXiv  Detail & Related papers  (2020-02-24T15:49:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.