Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction

• URL: http://arxiv.org/abs/2003.08802v1
• Date: Tue, 17 Mar 2020 02:49:51 GMT
• Title: Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction
• Authors: Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, Qi Tian
• Abstract summary: We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The model is action-category-agnostic and follows an encoder-decoder framework. The proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions.
• Score: 102.9787019197379
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human body for motion feature learning. This multiscale graph is adaptive during training and dynamic across network layers. Based on this graph, we propose a multiscale graph computational unit (MGCU) to extract features at individual scales and fuse features across scales. The entire model is action-category-agnostic and follows an encoder-decoder framework. The encoder consists of a sequence of MGCUs to learn motion features. The decoder uses a proposed graph-based gate recurrent unit to generate future poses. Extensive experiments show that the proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions on the datasets of Human 3.6M and CMU Mocap. We further investigate the learned multiscale graphs for the interpretability. The codes could be downloaded from https://github.com/limaosen0/DMGNN.

Related papers

• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• Multiscale Residual Learning of Graph Convolutional Sequence Chunks for Human Motion Prediction [23.212848643552395]
  A new method is proposed for human motion prediction by learning temporal and spatial dependencies. Our proposed method is able to effectively model the sequence information for motion prediction and outperform other techniques to set a new state-of-the-art.
  arXiv  Detail & Related papers  (2023-08-31T15:23:33Z)
• Multi-Graph Convolution Network for Pose Forecasting [0.8057006406834467]
  We propose a novel approach called the multi-graph convolution network (MGCN) for 3D human pose forecasting. MGCN simultaneously captures spatial and temporal information by introducing an augmented graph for pose sequences. In our evaluation, MGCN outperforms the state-of-the-art in pose prediction.
  arXiv  Detail & Related papers  (2023-04-11T03:59:43Z)
• Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction [120.08257447708503]
  Graph convolutional network based methods that model the body-joints' relations, have recently shown great promise in 3D skeleton-based human motion prediction. We propose a novel skeleton-parted graph scattering network (SPGSN) SPGSN outperforms state-of-the-art methods by remarkable margins of 13.8%, 9.3% and 2.7% in terms of 3D mean per joint position error (MPJPE) on Human3.6M, CMU Mocap and 3DPW datasets, respectively.
  arXiv  Detail & Related papers  (2022-07-31T05:51:39Z)
• DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction [8.142947808507365]
  We propose a feed-forward deep neural network for motion prediction. The entire model is suitable for all actions and follows a framework of encoder-decoder. Our approach outperforms SOTA methods on the datasets of Human3.6M and CMU Mocap.
  arXiv  Detail & Related papers  (2021-12-20T07:07:03Z)
• Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs [67.29552662707516]
  We propose a deeptemporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones. By the design, Skeleton-Graph predicts the future 3D poses without divergence on the long-term unlike prior works. Our results show an FDE improvement of at least 27% and an ADE of 4% on both the GTA-IM and PROX datasets respectively.
  arXiv  Detail & Related papers  (2021-09-21T15:33:40Z)
• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images [79.70127290464514]
  We decompose the task into two stages, i.e. person localization and pose estimation. And we propose three task-specific graph neural networks for effective message passing. Our approach achieves state-of-the-art performance on CMU Panoptic and Shelf datasets.
  arXiv  Detail & Related papers  (2021-09-13T11:44:07Z)
• Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction [92.16318571149553]
  We propose a multiscale-temporal graph neural network (MST-GNN) to predict the future 3D-based skeleton human poses. The MST-GNN outperforms state-of-the-art methods in both short and long-term motion prediction.
  arXiv  Detail & Related papers  (2021-08-25T14:05:37Z)
• MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction [34.565986275769745]
  We propose a novel Multi-Scale Residual Graph Convolution Network (MSR-GCN) for human pose prediction task. Our proposed approach is evaluated on two standard benchmark datasets, i.e., the Human3.6M dataset and the CMU Mocap dataset.
  arXiv  Detail & Related papers  (2021-08-16T15:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.