W-HMR: Human Mesh Recovery in World Space with Weak-supervised Camera Calibration and Orientation Correction

• URL: http://arxiv.org/abs/2311.17460v3
• Date: Mon, 25 Mar 2024 03:21:39 GMT
• Title: W-HMR: Human Mesh Recovery in World Space with Weak-supervised Camera Calibration and Orientation Correction
• Authors: Wei Yao, Hongwen Zhang, Yunlian Sun, Jinhui Tang,
• Abstract summary: We propose W-HMR, which decouples global body recovery into camera calibration, local body recovery, and global body orientation correction. We design the first weak-supervised camera calibration method for body distortion, eliminating dependence on focal length labels. We propose a novel orientation correction module to allow the reconstructed human body to remain normal in world space.
• Score: 35.42718669331158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For a long time, in reconstructing 3D human bodies from monocular images, most methods opted to simplify the task by minimizing the influence of the camera. Using a coarse focal length setting results in the reconstructed bodies not aligning well with distorted images. Ignoring camera rotation leads to an unrealistic reconstructed body pose in world space. Consequently, the application scenarios of existing methods are confined to controlled environments. When confronted with complex and diverse in-the-wild images, they struggle to achieve accurate and reasonable reconstruction in world space. To address the above issues, we propose W-HMR, which decouples global body recovery into camera calibration, local body recovery, and global body orientation correction. We design the first weak-supervised camera calibration method for body distortion, eliminating dependence on focal length labels and achieving finer mesh-image alignment. We propose a novel orientation correction module to allow the reconstructed human body to remain normal in world space. Decoupling body orientation and body pose enables our model to consider the accuracy in camera coordinate and the reasonableness in world coordinate simultaneously, expanding the range of applications. As a result, W-HMR achieves high-quality reconstruction in dual coordinate systems, particularly in challenging scenes. Codes and demos have been released on the project page https://yw0208.github.io/w-hmr/.

Related papers

• Simultaneous Map and Object Reconstruction [66.66729715211642]
  We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. We take inspiration from recent novel view synthesis methods and pose the reconstruction problem as a global optimization. By careful modeling of continuous-time motion, our reconstructions can compensate for the rolling shutter effects of rotating LiDAR sensors.
  arXiv  Detail & Related papers  (2024-06-19T23:53:31Z)
• WHAC: World-grounded Humans and Cameras [37.877565981937586]
  We aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly. We introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation. We present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras.
  arXiv  Detail & Related papers  (2024-03-19T17:58:02Z)
• Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction [66.10717041384625]
  Zolly is the first 3DHMR method focusing on perspective-distorted images. We propose a new camera model and a novel 2D representation, termed distortion image, which describes the 2D dense distortion scale of the human body. We extend two real-world datasets tailored for this task, all containing perspective-distorted human images.
  arXiv  Detail & Related papers  (2023-03-24T04:22:41Z)
• GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras [99.07219478953982]
  We present an approach for 3D global human mesh recovery from monocular videos recorded with dynamic cameras. We first propose a deep generative motion infiller, which autoregressively infills the body motions of occluded humans based on visible motions. In contrast to prior work, our approach reconstructs human meshes in consistent global coordinates even with dynamic cameras.
  arXiv  Detail & Related papers  (2021-12-02T18:59:54Z)
• Estimating Egocentric 3D Human Pose in Global Space [70.7272154474722]
  We present a new method for egocentric global 3D body pose estimation using a single-mounted fisheye camera. Our approach outperforms state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-04-27T20:01:57Z)
• PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videos [47.601288796052714]
  We develop two novel Pose frameworks, i.e., Serial PC-HMR and Parallel PC-HMR. Our frameworks are based on generic and complementary integration of data-driven learning and geometrical modeling. We perform extensive experiments on the popular bench-marks, i.e., Human3.6M, 3DPW and SURREAL, where our PC-HMR frameworks achieve the SOTA results.
  arXiv  Detail & Related papers  (2021-03-16T12:12:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.