TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos

• URL: http://arxiv.org/abs/2606.02350v1
• Date: Mon, 01 Jun 2026 15:00:18 GMT
• Title: TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos
• Abstract summary: TROPHIES aims to jointly estimate dynamic humans, static scenes, and camera poses in one global coordinate frame.<n>A global alignment and optimization module couples both branches by enforcing scale consistency, contact priors, and cross-view temporal coherence.<n>Experiments on EgoHuman and EgoExo4D demonstrate that TROPHIES consistently outperforms existing paradigms in both global fidelity and human-scene consistency.
• Score: 13.556917488868661
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing humans and their surrounding environments in a globally consistent 4D space is essential for comprehensive perception. However, prior works typically assume single-view inputs or decouple humans, scenes, and cameras, making them unable to recover coherent geometry, stable motion, and physically aligned trajectories. These limitations motivate us to introduce a new task: unified human-scene-camera reconstruction from multi-view videos, which aims to jointly estimate dynamic humans, static scenes, and camera poses in one global coordinate frame. We propose TROPHIES--Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos-a unified framework tailored for this task. TROPHIES features a Human Branch that models humans through temporal and spatial reasoning, and a Scene Branch that reconstructs static geometry with human-aware attention. A global alignment and optimization module couples both branches by enforcing scale consistency, contact priors, and cross-view temporal coherence. Experiments on EgoHuman and EgoExo4D demonstrate that TROPHIES achieves globally aligned, physically plausible 4D reconstructions and consistently outperforms existing paradigms in both global fidelity and human-scene consistency.

Related papers

• ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video [51.951128822418774]
  ReViV is a framework for holistic egocentric 4D reconstruction.<n>Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture.
  arXiv  Detail & Related papers  (2026-07-20T10:22:53Z)
• GUSH3R: Everyone Everywhere All at Once as Gaussians [30.884035795245946]
  We present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction.<n>From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass.<n>Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality.
  arXiv  Detail & Related papers  (2026-07-06T15:53:09Z)
• CoGS: Compositional Dynamic Human-Object Scenes Gaussian Splatting from Monocular Video [3.3660382321886217]
  We present a compositional Gaussian-splatting framework for monocular human-object reconstruction.<n>CoGS decomposes the video into three coordinated branches: an articulated human scene from a complete canonical prior, a rigid object field driven by an estimated object trajectory, and a static scene field regularized by weak scene-only planar primitives.<n>A six-stage optimization schedule first stabilizes the human and object independently, then fuses them with the scene under full-image supervision, visibility-aware human anchoring, object silhouette and motion constraints, and delayed scene regularization.
  arXiv  Detail & Related papers  (2026-06-27T09:10:22Z)
• UniCon3R: Contact-aware 3D Human-Scene Reconstruction from Monocular Video [82.5562736830041]
  We introduce UniCon3R, a unified feed-forward framework for online human-scene 4D reconstruction from monocular videos.<n>We explicitly model interaction by inferring 3D contact from the human pose and scene geometry.<n>This enables UniCon3R to jointly recover high-fidelity scene geometry and spatially aligned 3D humans within the scene.
  arXiv  Detail & Related papers  (2026-04-21T19:06:50Z)
• EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agents [85.77432303199176]
  We propose EmbodMocap, a portable and affordable data collection pipeline using two moving iPhones.<n>Our key idea is to jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes.<n>Based on the collected data, we empower three embodied AI tasks: monocular human-scene-reconstruction, where we fine-tune feedforward models that output metric-scale, world-space aligned humans and scenes; physics-based character animation, where we prove our data could be used to scale human-object interaction skills and scene-aware motion tracking; and robot motion control, where we train a humanoid robot via
  arXiv  Detail & Related papers  (2026-02-26T16:53:41Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• WATCH: World-aware Allied Trajectory and pose reconstruction for Camera and Human [14.608329202942057]
  Global human motion reconstruction from in-the-wild monocular videos is increasingly demanded across VR, graphics, and robotics applications.<n>We present WATCH (World-aware Allied Trajectory and pose reconstruction for Camera and Human), a unified framework addressing both challenges.<n>Our work demonstrates the effectiveness of jointly modeling camera-human motion relationships and offers new insights for addressing the long-standing challenge of camera translation integration in global human motion reconstruction.
  arXiv  Detail & Related papers  (2025-09-04T18:29:48Z)
• Reconstructing People, Places, and Cameras [57.81696692335401]
  "Humans and Structure from Motion" (HSfM) is a method for jointly reconstructing multiple human meshes, scene point clouds, and camera parameters in a metric world coordinate system.<n>Our results show that incorporating human data into the SfM pipeline improves camera pose estimation.
  arXiv  Detail & Related papers  (2024-12-23T18:58:34Z)
• Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer [38.85054820740242]
  We present a novel approach for generating high-quality, coherent human videos from a single image. Our framework combines the strengths of diffusion transformers for capturing global correlations and CNNs for accurate condition injection. We demonstrate our method's ability to synthesize 360-degree realistic, coherent human motion videos.
  arXiv  Detail & Related papers  (2024-05-27T17:53:29Z)
• GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras [99.07219478953982]
  We present an approach for 3D global human mesh recovery from monocular videos recorded with dynamic cameras. We first propose a deep generative motion infiller, which autoregressively infills the body motions of occluded humans based on visible motions. In contrast to prior work, our approach reconstructs human meshes in consistent global coordinates even with dynamic cameras.
  arXiv  Detail & Related papers  (2021-12-02T18:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.