SVMA: A GAN-based model for Monocular 3D Human Pose Estimation

• URL: http://arxiv.org/abs/2106.05616v1
• Date: Thu, 10 Jun 2021 09:43:57 GMT
• Title: SVMA: A GAN-based model for Monocular 3D Human Pose Estimation
• Authors: Yicheng Deng, Yongqi Sun, Jiahui Zhu
• Abstract summary: We present an unsupervised GAN-based model to recover 3D human pose from 2D joint locations extracted from a single image. Considering the reprojection constraint, our model can estimate the camera so that we can reproject the estimated 3D pose to the original 2D pose. Results on Human3.6M show that our method outperforms all the state-of-the-art methods, and results on MPI-INF-3DHP show that our method outperforms state-of-the-art by approximately 15.0%.
• Score: 0.8379286663107844
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Recovering 3D human pose from 2D joints is a highly unconstrained problem, especially without any video or multi-view information. We present an unsupervised GAN-based model to recover 3D human pose from 2D joint locations extracted from a single image. Our model uses a GAN to learn the mapping of distribution from 2D poses to 3D poses, not the simple 2D-3D correspondence. Considering the reprojection constraint, our model can estimate the camera so that we can reproject the estimated 3D pose to the original 2D pose. Based on this reprojection method, we can rotate and reproject the generated pose to get our "new" 2D pose and then use a weight sharing generator to estimate the "new" 3D pose and a "new" camera. Through the above estimation process, we can define the single-view-multi-angle consistency loss during training to simulate multi-view consistency, which means the 3D poses and cameras estimated from two angles of a single view should be able to be mixed to generate rich 2D reprojections, and the 2D reprojections reprojected from the same 3D pose should be consistent. The experimental results on Human3.6M show that our method outperforms all the state-of-the-art methods, and results on MPI-INF-3DHP show that our method outperforms state-of-the-art by approximately 15.0%.

Related papers

• MPL: Lifting 3D Human Pose from Multi-view 2D Poses [75.26416079541723]
  We propose combining 2D pose estimation, for which large and rich training datasets exist, and 2D-to-3D pose lifting, using a transformer-based network. Our experiments demonstrate decreases up to 45% in MPJPE errors compared to the 3D pose obtained by triangulating the 2D poses.
  arXiv  Detail & Related papers  (2024-08-20T12:55:14Z)
• ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation [54.86887812687023]
  Most 3D-HPE methods rely on regression models, which assume a one-to-one mapping between inputs and outputs. We propose ManiPose, a novel manifold-constrained multi-hypothesis model capable of proposing multiple candidate 3D poses for each 2D input. Unlike previous multi-hypothesis approaches, our solution is completely supervised and does not rely on complex generative models.
  arXiv  Detail & Related papers  (2023-12-11T13:50:10Z)
• Unsupervised Multi-Person 3D Human Pose Estimation From 2D Poses Alone [4.648549457266638]
  We present one of the first studies investigating the feasibility of unsupervised multi-person 2D-3D pose estimation. Our method involves independently lifting each subject's 2D pose to 3D, before combining them in a shared 3D coordinate system. This by itself enables us to retrieve an accurate 3D reconstruction of their poses.
  arXiv  Detail & Related papers  (2023-09-26T11:42:56Z)
• MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling [59.74064212110042]
  mpmcan handle multiple tasks including 3D human pose estimation, 3D pose estimation from cluded 2D pose, and 3D pose completion in a textocbfsingle framework. We conduct extensive experiments and ablation studies on several widely used human pose datasets and achieve state-of-the-art performance on MPI-INF-3DHP.
  arXiv  Detail & Related papers  (2023-06-29T10:30:00Z)
• CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations [25.05308239278207]
  We present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity. We also introduce a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators.
  arXiv  Detail & Related papers  (2023-01-08T05:07:41Z)
• SPGNet: Spatial Projection Guided 3D Human Pose Estimation in Low Dimensional Space [14.81199315166042]
  We propose a method for 3D human pose estimation that mixes multi-dimensional re-projection into supervised learning. Based on the estimation results for the dataset Human3.6M, our approach outperforms many state-of-the-art methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-06-04T00:51:00Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach [76.10879433430466]
  We propose to estimate 3D human pose from multi-view images and a few IMUs attached at person's limbs. It operates by firstly detecting 2D poses from the two signals, and then lifting them to the 3D space. The simple two-step approach reduces the error of the state-of-the-art by a large margin on a public dataset.
  arXiv  Detail & Related papers  (2020-03-25T00:26:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.