Scene and Human in One World: Reconstruction in a Feedforward Pass

• URL: http://arxiv.org/abs/2606.27720v1
• Date: Fri, 26 Jun 2026 05:08:55 GMT
• Title: Scene and Human in One World: Reconstruction in a Feedforward Pass
• Abstract summary: Show is a mask-promptable human mesh recovery framework that couples feed-forward 3D scene reconstruction with Human Mesh Recovery.<n> SHOW injects human semantics and scale priors from parametric human models into normalized point-map prediction.<n>It constrains human mesh estimation, encouraging spatially consistent human placement and improved human-scene alignment.
• Score: 32.88487546112625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconstructing humans in dynamic scenes from moving monocular cameras remains challenging due to scale ambiguity, human-scene misalignment, and occlusion interference. Rather than treating human mesh recovery and scene reconstruction as separate tasks, we believe that accurate human-scene reconstruction requires the two tasks to mutually inform each other: parametric human models offer semantic structure and metric-scale priors, while scene geometry provides spatial context for human localization and alignment. Built on this insight, we introduce SHOW, a mask-promptable human mesh recovery framework that couples feed-forward 3D scene reconstruction with Human Mesh Recovery in a unified metric space. SHOW injects human semantics and scale priors from parametric human models into normalized point-map prediction, enabling metric-scale scene reconstruction from inherently scale-ambiguous monocular input. In turn, the recovered scene geometry constrains human mesh estimation, encouraging spatially consistent human placement and improved human-scene alignment. To handle complex multi-person and cluttered scenes, SHOW further incorporates a promptable masking mechanism that enables flexible target-human selection while suppressing background distractions and occlusion interference. Through joint training, the model learns both human-aware geometric features and geometry-constrained human features, producing aligned metric-scale reconstructions from monocular human-centric videos. Extensive experiments demonstrate that SHOW improves metric-scale consistency, human-scene alignment, and reconstruction accuracy under challenging camera motion, occlusion, and cluttered backgrounds.

Related papers

• Feed-Forward Multi-view Multi-person Reconstruction with Contrastive Human-Aware 3D Representation [12.32801567231421]
  We propose a new top-down paradigm that maintains a unified, instance-centric human-aware 3D space.<n>Cross-modal contrastive learning enables simultaneous camera calibration, cross-view association, and human reconstruction.<n>Experiments demonstrate robust, accurate, and efficient multi-view human reconstruction in challenging real-world scenarios.
  arXiv  Detail & Related papers  (2026-09-01T05:24:20Z)
• HumanSplatHMR: Closing the Loop Between Human Mesh Recovery and Gaussian Splatting Avatar [19.538398405782996]
  Accurately recovering human pose and appearance from video is an essential component of scene reconstruction.<n>This paper proposes HumanSplatHMR, a joint optimization framework that refines 3D human poses.<n> Experiments show consistent improvements over pose recovery baselines that decouple pose estimation from avatar reconstruction.
  arXiv  Detail & Related papers  (2026-05-04T16:26:11Z)
• AHAP: Reconstructing Arbitrary Humans from Arbitrary Perspectives with Geometric Priors [81.50960055126156]
  We present textbfAHAP, a feed-forward framework for reconstructing arbitrary humans from arbitrary camera perspectives.<n>Our core lies in the effective fusion of multi-view geometry to assist human association, reconstruction and localization.<n>A Human Head fuses cross-view features and scene context for SMPL prediction, guided by cross-view reprojection losses to enforce body pose consistency.
  arXiv  Detail & Related papers  (2026-02-27T11:53:45Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitives [65.89192712575797]
  We introduce CRISP, a method that recovers simulatable human motion and scene geometry from monocular video.<n>Our approach reduces motion tracking failure rates from 55.2% to 6.9% on human-centric video benchmarks.<n>This demonstrates CRISP's ability to generate physically-valid human motion and interaction environments at scale.
  arXiv  Detail & Related papers  (2025-12-16T18:59:50Z)
• HuPrior3R: Incorporating Human Priors for Better 3D Dynamic Reconstruction from Monocular Videos [20.256869569776118]
  We propose to incorporate hybrid geometric priors that combine SMPL human body models with monocular depth estimation.<n>HuPrior3R, featuring a hierarchical pipeline with refinement components, then applies strategic cropping and cross-attention fusion for human-specific detail enhancement.<n>Experiments on TUM Dynamics and GTA-IM datasets demonstrate superior performance in dynamic human reconstruction.
  arXiv  Detail & Related papers  (2025-12-06T09:42:56Z)
• Dynamic Avatar-Scene Rendering from Human-centric Context [75.95641456716373]
  We propose bf Separate-then-Map (StM) strategy to bridge separately defined and optimized models.<n>StM significantly outperforms existing state-of-the-art methods in both visual quality and rendering accuracy.
  arXiv  Detail & Related papers  (2025-11-13T17:39:06Z)
• HumanCrafter: Synergizing Generalizable Human Reconstruction and Semantic 3D Segmentation [51.27178551863772]
  We propose a unified framework that enables the joint modeling of appearance and human-part semantics from a single image.<n>HumanCrafter surpasses existing state-of-the-art methods in both 3D human-part segmentation and 3D human reconstruction from a single image.
  arXiv  Detail & Related papers  (2025-11-01T09:29:36Z)
• SHARE: Scene-Human Aligned Reconstruction [10.764401463569442]
  We introduce Scene-Human Aligned REconstruction, a technique that leverages the scene geometry's inherent spatial cues to accurately ground human motion reconstruction.<n>It iteratively refines the human's positions at theses by comparing the human mesh against a human point map extracted from the scene using the mask.<n>Our approach enables more accurate 3D human placement while reconstructing the surrounding scene, facilitating use cases on both curated datasets and in-the-wild web videos.
  arXiv  Detail & Related papers  (2025-10-17T06:12:10Z)
• ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos [18.73641648585445]
  Recent neural rendering advances have enabled holistic human-scene reconstruction but require pre-calibrated camera and human poses.<n>We introduce a novel unified framework that simultaneously performs camera tracking, human pose estimation and human-scene reconstruction in an online fashion.<n>Specifically, we design a human deformation module to reconstruct the details and enhance generalizability to out-of-distribution poses faithfully.
  arXiv  Detail & Related papers  (2025-04-17T17:59:02Z)
• Realistic Clothed Human and Object Joint Reconstruction from a Single Image [26.57698106821237]
  We introduce a novel implicit approach for jointly reconstructing realistic 3D clothed humans and objects from a monocular view.<n>For the first time, we model both the human and the object with an implicit representation, allowing to capture more realistic details such as clothing.
  arXiv  Detail & Related papers  (2025-02-25T12:26:36Z)
• Reconstructing People, Places, and Cameras [57.81696692335401]
  "Humans and Structure from Motion" (HSfM) is a method for jointly reconstructing multiple human meshes, scene point clouds, and camera parameters in a metric world coordinate system.<n>Our results show that incorporating human data into the SfM pipeline improves camera pose estimation.
  arXiv  Detail & Related papers  (2024-12-23T18:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.