AMPose: Alternately Mixed Global-Local Attention Model for 3D Human Pose Estimation

• URL: http://arxiv.org/abs/2210.04216v5
• Date: Tue, 31 Oct 2023 12:46:21 GMT
• Title: AMPose: Alternately Mixed Global-Local Attention Model for 3D Human Pose Estimation
• Authors: Hongxin Lin, Yunwei Chiu and Peiyuan Wu
• Abstract summary: We propose a novel method to combine the global and physically connected relations among joints towards 3D human pose estimation. In the AMPose, the Transformer encoder is applied to connect each joint with all the other joints, while GCNs are applied to capture information on physically connected relations. Our model also shows better generalization ability by testing on the MPI-INF-3DHP dataset.
• Score: 2.9823712604345993
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The graph convolutional networks (GCNs) have been applied to model the physically connected and non-local relations among human joints for 3D human pose estimation (HPE). In addition, the purely Transformer-based models recently show promising results in video-based 3D HPE. However, the single-frame method still needs to model the physically connected relations among joints because the feature representations transformed only by global relations via the Transformer neglect information on the human skeleton. To deal with this problem, we propose a novel method in which the Transformer encoder and GCN blocks are alternately stacked, namely AMPose, to combine the global and physically connected relations among joints towards HPE. In the AMPose, the Transformer encoder is applied to connect each joint with all the other joints, while GCNs are applied to capture information on physically connected relations. The effectiveness of our proposed method is evaluated on the Human3.6M dataset. Our model also shows better generalization ability by testing on the MPI-INF-3DHP dataset. Code can be retrieved at https://github.com/erikervalid/AMPose.

Related papers

• MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network [2.7268855969580166]
  We present a novel Attention-GCNFormer block that divides the number of channels by using two parallel transformer and GCNFormer streams. Our proposed GCNFormer module exploits the local relationship between adjacent joints, outputting a new representation that is complementary to the transformer output. We evaluate our model on two popular benchmark datasets: Human3.6M and MPI-INF-3DHP.
  arXiv  Detail & Related papers  (2023-10-25T01:46:35Z)
• Joint-Relation Transformer for Multi-Person Motion Prediction [79.08243886832601]
  We propose the Joint-Relation Transformer to enhance interaction modeling. Our method achieves a 13.4% improvement of 900ms VIM on 3DPW-SoMoF/RC and 17.8%/12.0% improvement of 3s MPJPE.
  arXiv  Detail & Related papers  (2023-08-09T09:02:47Z)
• Iterative Graph Filtering Network for 3D Human Pose Estimation [5.177947445379688]
  Graph convolutional networks (GCNs) have proven to be an effective approach for 3D human pose estimation. In this paper, we introduce an iterative graph filtering framework for 3D human pose estimation. Our approach builds upon the idea of iteratively solving graph filtering with Laplacian regularization.
  arXiv  Detail & Related papers  (2023-07-29T20:46:44Z)
• Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation [53.04781510348416]
  Video-based 3D human pose and shape estimations are evaluated by intra-frame accuracy and inter-frame smoothness. We propose to structurally decouple the modeling of long-term and short-term correlations in an end-to-end framework, Global-to-Local Transformer (GLoT) Our GLoT surpasses previous state-of-the-art methods with the lowest model parameters on popular benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M.
  arXiv  Detail & Related papers  (2023-03-26T14:57:49Z)
• (Fusionformer):Exploiting the Joint Motion Synergy with Fusion Network Based On Transformer for 3D Human Pose Estimation [1.52292571922932]
  Many previous methods lack the understanding of local joint information.cite8888987considers the temporal relationship of a single joint in this work. Our proposed textbfFusionformer method introduces a global-temporal self-trajectory module and a cross-temporal self-trajectory module. The results show an improvement of 2.4% MPJPE and 4.3% P-MPJPE on the Human3.6M dataset.
  arXiv  Detail & Related papers  (2022-10-08T12:22:10Z)
• Jointformer: Single-Frame Lifting Transformer with Error Prediction and Refinement for 3D Human Pose Estimation [11.592567773739407]
  3D human pose estimation technologies have the potential to greatly increase the availability of human movement data. The best-performing models for single-image 2D-3D lifting use graph convolutional networks (GCNs) that typically require some manual input to define the relationships between different body joints. We propose a novel transformer-based approach that uses the more generalised self-attention mechanism to learn these relationships.
  arXiv  Detail & Related papers  (2022-08-07T12:07:19Z)
• MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video [75.23812405203778]
  Recent solutions have been introduced to estimate 3D human pose from 2D keypoint sequence by considering body joints among all frames globally to learn-temporal correlation. We propose Mix Mix, which has temporal transformer block to separately model the temporal motion of each joint and a transformer block inter-joint spatial correlation. In addition, the network output is extended from the central frame to entire frames of input video, improving the coherence between the input and output benchmarks.
  arXiv  Detail & Related papers  (2022-03-02T04:20:59Z)
• Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
  The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies. We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
  arXiv  Detail & Related papers  (2021-12-14T13:14:24Z)
• THUNDR: Transformer-based 3D HUmaN Reconstruction with Markers [67.8628917474705]
  THUNDR is a transformer-based deep neural network methodology to reconstruct the 3d pose and shape of people. We show state-of-the-art results on Human3.6M and 3DPW, for both the fully-supervised and the self-supervised models. We observe very solid 3d reconstruction performance for difficult human poses collected in the wild.
  arXiv  Detail & Related papers  (2021-06-17T09:09:24Z)
• 3D Human Pose Estimation with Spatial and Temporal Transformers [59.433208652418976]
  We present PoseFormer, a purely transformer-based approach for 3D human pose estimation in videos. Inspired by recent developments in vision transformers, we design a spatial-temporal transformer structure. We quantitatively and qualitatively evaluate our method on two popular and standard benchmark datasets.
  arXiv  Detail & Related papers  (2021-03-18T18:14:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.