Egocentric Whole-Body Human Mesh Recovery with Prior-Guided Learning

• URL: http://arxiv.org/abs/2605.08606v1
• Date: Sat, 09 May 2026 01:59:01 GMT
• Title: Egocentric Whole-Body Human Mesh Recovery with Prior-Guided Learning
• Abstract summary: We propose a prior-guided learning framework that reconstructs whole-body meshes from a single egocentric image.<n> Experiments across multiple egocentric benchmarks demonstrate improved whole-body reconstruction compared to state-of-the-art methods.
• Score: 4.1709348827585515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric human mesh recovery (HMR) from monocular head-mounted cameras is increasingly important for AR/VR applications, but remains challenging due to the lack of reliable ground-truth (GT) annotations based on parametric human body models such as SMPL and SMPL-X for real egocentric images. Existing egocentric HMR methods typically rely on pseudo-GT and focus on body pose estimation, which limits their ability to recover fine-grained whole-body details such as hands and face. We study egocentric whole-body human mesh recovery and propose a prior-guided learning framework that reconstructs whole-body meshes from a single egocentric image. We construct more accurate optimization-based pseudo-GT aligned with 3D joint supervision, and leverage multiple priors by adapting an exocentric HMR foundation model together with a diffusion-based pose prior. A deterministic undistortion module is further adopted to handle fisheye distortions in egocentric images. Experiments across multiple egocentric benchmarks demonstrate improved whole-body reconstruction compared to state-of-the-art methods, and show that our optimization-based pseudo-GT is substantially more accurate than existing regression-based pseudo-GT. To facilitate reproducibility, the code and dataset annotations are publicly available at https://github.com/naso06/EgoSMPLX.

Related papers

• Anny-Fit: All-Age Human Mesh Recovery [24.422315218120932]
  Anny-Fit is a camera-space optimization framework for all-age 3D human mesh recovery.<n>Our approach bridges adult-only and all-age modeling by enabling zero-shot adaptation of adult-trained HMR pipelines to the full age spectrum without retraining.
  arXiv  Detail & Related papers  (2026-05-06T10:23:10Z)
• AHAP: Reconstructing Arbitrary Humans from Arbitrary Perspectives with Geometric Priors [81.50960055126156]
  We present textbfAHAP, a feed-forward framework for reconstructing arbitrary humans from arbitrary camera perspectives.<n>Our core lies in the effective fusion of multi-view geometry to assist human association, reconstruction and localization.<n>A Human Head fuses cross-view features and scene context for SMPL prediction, guided by cross-view reprojection losses to enforce body pose consistency.
  arXiv  Detail & Related papers  (2026-02-27T11:53:45Z)
• VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery [75.62565146049015]
  We introduce a dual-memory augmented HMR critique agent with self-reflection to produce context-aware quality scores for predicted meshes.<n>These scores distill fine-grained cues about 3D human motion structure, physical feasibility, and alignment with the input image.<n>We propose a group preference alignment framework for finetuning diffusion-based HMR models.
  arXiv  Detail & Related papers  (2026-02-22T13:19:06Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• DPHMs: Diffusion Parametric Head Models for Depth-based Tracking [42.016598097736626]
  We introduce Diffusion Parametric Head Models (DPHMs) DPHMs are a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences. We propose a latent diffusion-based prior to regularize volumetric head reconstruction and tracking.
  arXiv  Detail & Related papers  (2023-12-02T08:34:22Z)
• W-HMR: Monocular Human Mesh Recovery in World Space with Weak-Supervised Calibration [57.37135310143126]
  Previous methods for 3D motion recovery from monocular images often fall short due to reliance on camera coordinates. We introduce W-HMR, a weak-supervised calibration method that predicts "reasonable" focal lengths based on body distortion information. We also present the OrientCorrect module, which corrects body orientation for plausible reconstructions in world space.
  arXiv  Detail & Related papers  (2023-11-29T09:02:07Z)
• 1st Place Solution of Egocentric 3D Hand Pose Estimation Challenge 2023 Technical Report:A Concise Pipeline for Egocentric Hand Pose Reconstruction [11.551318550321938]
  Using AssemblyHands, this challenge focuses on egocentric 3D hand pose estimation from a single-view image. We adopt ViT based backbones and a simple regressor for 3D keypoints prediction, which provides strong model baselines. Our method achieved 12.21mm MPJPE on test dataset, achieve the first place in Egocentric 3D Hand Pose Estimation challenge.
  arXiv  Detail & Related papers  (2023-10-07T10:25:50Z)
• Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction [66.10717041384625]
  Zolly is the first 3DHMR method focusing on perspective-distorted images. We propose a new camera model and a novel 2D representation, termed distortion image, which describes the 2D dense distortion scale of the human body. We extend two real-world datasets tailored for this task, all containing perspective-distorted human images.
  arXiv  Detail & Related papers  (2023-03-24T04:22:41Z)
• PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images [60.33197938330409]
  PyMAF-X is a regression-based approach to recovering parametric full-body models from monocular images. PyMAF and PyMAF-X effectively improve the mesh-image alignment and achieve new state-of-the-art results.
  arXiv  Detail & Related papers  (2022-07-13T17:58:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.