SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation

• URL: http://arxiv.org/abs/2404.15276v1
• Date: Tue, 23 Apr 2024 17:59:59 GMT
• Title: SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation
• Authors: Xiangyu Xu, Lijuan Liu, Shuicheng Yan,
• Abstract summary: We propose an SMPL-based Transformer framework (SMPLer) to address this issue. SMPLer incorporates two key ingredients: a decoupled attention operation and an SMPL-based target representation. Extensive experiments demonstrate the effectiveness of SMPLer against existing 3D human shape and pose estimation methods.
• Score: 74.07836010698801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing Transformers for monocular 3D human shape and pose estimation typically have a quadratic computation and memory complexity with respect to the feature length, which hinders the exploitation of fine-grained information in high-resolution features that is beneficial for accurate reconstruction. In this work, we propose an SMPL-based Transformer framework (SMPLer) to address this issue. SMPLer incorporates two key ingredients: a decoupled attention operation and an SMPL-based target representation, which allow effective utilization of high-resolution features in the Transformer. In addition, based on these two designs, we also introduce several novel modules including a multi-scale attention and a joint-aware attention to further boost the reconstruction performance. Extensive experiments demonstrate the effectiveness of SMPLer against existing 3D human shape and pose estimation methods both quantitatively and qualitatively. Notably, the proposed algorithm achieves an MPJPE of 45.2 mm on the Human3.6M dataset, improving upon Mesh Graphormer by more than 10% with fewer than one-third of the parameters. Code and pretrained models are available at https://github.com/xuxy09/SMPLer.

Related papers

• OccLoff: Learning Optimized Feature Fusion for 3D Occupancy Prediction [5.285847977231642]
  3D semantic occupancy prediction is crucial for ensuring the safety in autonomous driving. Existing fusion-based occupancy methods typically involve performing a 2D-to-3D view transformation on image features. We propose OccLoff, a framework that Learns to optimize Feature Fusion for 3D occupancy prediction.
  arXiv  Detail & Related papers  (2024-11-06T06:34:27Z)
• MonoMM: A Multi-scale Mamba-Enhanced Network for Real-time Monocular 3D Object Detection [9.780498146964097]
  We propose an innovative network architecture, MonoMM, for real-time monocular 3D object detection. MonoMM consists of Focused Multi-Scale Fusion (FMF) and Depth-Aware Feature Enhancement Mamba (DMB) modules. Our method outperforms previous monocular methods and achieves real-time detection.
  arXiv  Detail & Related papers  (2024-08-01T10:16:58Z)
• SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation [53.675725490807615]
  We introduce SDPose, a new self-distillation method for improving the performance of small transformer-based models. SDPose-T obtains 69.7% mAP with 4.4M parameters and 1.8 GFLOPs, while SDPose-S-V2 obtains 73.5% mAP on the MSCOCO validation dataset.
  arXiv  Detail & Related papers  (2024-04-04T15:23:14Z)
• EVOPOSE: A Recursive Transformer For 3D Human Pose Estimation With Kinematic Structure Priors [72.33767389878473]
  We propose a transformer-based model EvoPose to introduce the human body prior knowledge for 3D human pose estimation effectively. A Structural Priors Representation (SPR) module represents human priors as structural features carrying rich body patterns. A Recursive Refinement (RR) module is applied to the 3D pose outputs by utilizing estimated results and further injects human priors simultaneously.
  arXiv  Detail & Related papers  (2023-06-16T04:09:16Z)
• A Modular Multi-stage Lightweight Graph Transformer Network for Human Pose and Shape Estimation from 2D Human Pose [4.598337780022892]
  We introduce a pose-based human mesh reconstruction approach that prioritizes computational efficiency without sacrificing reconstruction accuracy. Our method consists of a 2D-to-3D lifter module that utilizes graph transformers to analyze structured and implicit joint correlations in 2D human poses, and a mesh regression module that combines the extracted pose features with a mesh template to produce the final human mesh parameters.
  arXiv  Detail & Related papers  (2023-01-31T04:42:47Z)
• View-Disentangled Transformer for Brain Lesion Detection [50.4918615815066]
  We propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection. First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan. Second, the transformer models a stack of slice features as multiple 2D views and enhance these features view-by-view. Third, we deploy the proposed transformer module in a transformer backbone, which can effectively detect the 2D regions surrounding brain lesions.
  arXiv  Detail & Related papers  (2022-09-20T11:58:23Z)
• Learnable human mesh triangulation for 3D human pose and shape estimation [6.699132260402631]
  The accuracy of joint rotation and shape estimation has received relatively little attention in the skinned multi-person linear model (SMPL)-based human mesh reconstruction from multi-view images. We propose a two-stage method to resolve the ambiguity of joint rotation and shape reconstruction and the difficulty of network learning. The proposed method significantly outperforms the previous works in terms of joint rotation and shape estimation, and achieves competitive performance in terms of joint location estimation.
  arXiv  Detail & Related papers  (2022-08-24T01:11:57Z)
• Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers [17.22112222736234]
  Transformer encoder architectures have recently achieved state-of-the-art results on monocular 3D human mesh reconstruction. Due to the large memory overhead and slow inference speed, it is difficult to deploy such models for practical use. We propose a novel transformer encoder-decoder architecture for 3D human mesh reconstruction from a single image, called FastMETRO.
  arXiv  Detail & Related papers  (2022-07-27T22:54:09Z)
• Geometry-Contrastive Transformer for Generalized 3D Pose Transfer [95.56457218144983]
  The intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. We propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies. We present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task.
  arXiv  Detail & Related papers  (2021-12-14T13:14:24Z)
• Direct Multi-view Multi-person 3D Pose Estimation [138.48139701871213]
  We present Multi-view Pose transformer (MvP) for estimating multi-person 3D poses from multi-view images. MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks. We show experimentally that our MvP model outperforms the state-of-the-art methods on several benchmarks while being much more efficient.
  arXiv  Detail & Related papers  (2021-11-07T13:09:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.