Multi-hop graph transformer network for 3D human pose estimation

• URL: http://arxiv.org/abs/2405.03055v1
• Date: Sun, 5 May 2024 21:29:20 GMT
• Title: Multi-hop graph transformer network for 3D human pose estimation
• Authors: Zaedul Islam, A. Ben Hamza,
• Abstract summary: We introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix. The integration of dilated convolutional layers enhances the model's ability to handle spatial generalizations required for accurate localization of the human body joints.
• Score: 4.696083734269233
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi-hop graph convolutional block comprised of multi-hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi-hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model's ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets.

Related papers

• Flexible graph convolutional network for 3D human pose estimation [4.696083734269233]
  We introduce Flex-GCN, a flexible graph convolutional network designed to learn graph representations that capture broader global information and dependencies. At its core is the flexible graph convolution, which aggregates features from both immediate and second-order neighbors of each node. Our network architecture comprises residual blocks of flexible graph convolutional layers, as well as a global response normalization layer for global feature aggregation, normalization and calibration.
  arXiv  Detail & Related papers  (2024-07-26T20:46:28Z)
• Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction [120.08257447708503]
  Graph convolutional network based methods that model the body-joints' relations, have recently shown great promise in 3D skeleton-based human motion prediction. We propose a novel skeleton-parted graph scattering network (SPGSN) SPGSN outperforms state-of-the-art methods by remarkable margins of 13.8%, 9.3% and 2.7% in terms of 3D mean per joint position error (MPJPE) on Human3.6M, CMU Mocap and 3DPW datasets, respectively.
  arXiv  Detail & Related papers  (2022-07-31T05:51:39Z)
• MUG: Multi-human Graph Network for 3D Mesh Reconstruction from 2D Pose [20.099670445427964]
  Reconstructing multi-human body mesh from a single monocular image is an important but challenging computer vision problem. In this work, through a single graph neural network, we construct coherent multi-human meshes using only multi-human 2D pose as input.
  arXiv  Detail & Related papers  (2022-05-25T08:54:52Z)
• Hierarchical Graph Networks for 3D Human Pose Estimation [50.600944798627786]
  Recent 2D-to-3D human pose estimation works tend to utilize the graph structure formed by the topology of the human skeleton. We argue that this skeletal topology is too sparse to reflect the body structure and suffer from serious 2D-to-3D ambiguity problem. We propose a novel graph convolution network architecture, Hierarchical Graph Networks, to overcome these weaknesses.
  arXiv  Detail & Related papers  (2021-11-23T15:09:03Z)
• Higher-Order Implicit Fairing Networks for 3D Human Pose Estimation [1.1501261942096426]
  We introduce a higher-order graph convolutional framework with initial residual connections for 2D-to-3D pose estimation. Our model is able to capture the long-range dependencies between body joints. Experiments and ablations studies conducted on two standard benchmarks demonstrate the effectiveness of our model.
  arXiv  Detail & Related papers  (2021-11-01T13:48:55Z)
• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images [79.70127290464514]
  We decompose the task into two stages, i.e. person localization and pose estimation. And we propose three task-specific graph neural networks for effective message passing. Our approach achieves state-of-the-art performance on CMU Panoptic and Shelf datasets.
  arXiv  Detail & Related papers  (2021-09-13T11:44:07Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)
• Graph Stacked Hourglass Networks for 3D Human Pose Estimation [1.0660480034605242]
  We propose a novel graph convolutional network architecture, Graph Stacked Hourglass Networks, for 2D-to-3D human pose estimation tasks. The proposed architecture consists of repeated encoder-decoder, in which graph-structured features are processed across three different scales of human skeletal representations.
  arXiv  Detail & Related papers  (2021-03-30T14:25:43Z)
• Unsupervised Cross-Modal Alignment for Multi-Person 3D Pose Estimation [52.94078950641959]
  We present a deployment friendly, fast bottom-up framework for multi-person 3D human pose estimation. We adopt a novel neural representation of multi-person 3D pose which unifies the position of person instances with their corresponding 3D pose representation. We propose a practical deployment paradigm where paired 2D or 3D pose annotations are unavailable.
  arXiv  Detail & Related papers  (2020-08-04T07:54:25Z)
• HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimation [54.23770284299979]
  This paper introduces a novel form of supervision - Hierarchical Multi-person Ordinal Relations (HMOR) HMOR encodes interaction information as the ordinal relations of depths and angles hierarchically. An integrated top-down model is designed to leverage these ordinal relations in the learning process. The proposed method significantly outperforms state-of-the-art methods on publicly available multi-person 3D pose datasets.
  arXiv  Detail & Related papers  (2020-08-01T07:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.