CoGS: Compositional Dynamic Human-Object Scenes Gaussian Splatting from Monocular Video

• URL: http://arxiv.org/abs/2606.28820v1
• Date: Sat, 27 Jun 2026 09:10:22 GMT
• Title: CoGS: Compositional Dynamic Human-Object Scenes Gaussian Splatting from Monocular Video
• Abstract summary: We present a compositional Gaussian-splatting framework for monocular human-object reconstruction.<n>CoGS decomposes the video into three coordinated branches: an articulated human scene from a complete canonical prior, a rigid object field driven by an estimated object trajectory, and a static scene field regularized by weak scene-only planar primitives.<n>A six-stage optimization schedule first stabilizes the human and object independently, then fuses them with the scene under full-image supervision, visibility-aware human anchoring, object silhouette and motion constraints, and delayed scene regularization.
• Score: 3.3660382321886217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconstructing dynamic human--object interaction scenes from monocular video is difficult because the human, manipulated object, and background obey different motion models while sharing the same pixels. Existing dynamic radiance-field and Gaussian-splatting methods often entangle these components, causing object motion to leak into the human or static scene, and monocular human reconstruction remains underconstrained in regions that are rarely observed. We present CoGS, a compositional Gaussian-splatting framework for monocular human--object scene reconstruction. CoGS decomposes the video into three coordinated branches: an articulated human initialized from a complete canonical prior, a rigid object field driven by an estimated object trajectory, and a static scene field regularized by weak scene-only planar primitives when available. A six-stage optimization schedule first stabilizes the human and object independently, then fuses them with the scene under full-image supervision, visibility-aware human anchoring, object silhouette and motion constraints, and delayed scene regularization. This design keeps each component responsible for its own geometry and motion while allowing photometric evidence to correct the final composite. Experiments on HOSNeRF and NeuMan show that CoGS improves both human--object interaction reconstruction and in-the-wild human--scene rendering, achieving stronger fidelity and perceptual quality across full-frame and human-focused evaluations. Code will be released upon publication.

Related papers

• GUSH3R: Everyone Everywhere All at Once as Gaussians [30.884035795245946]
  We present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction.<n>From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass.<n>Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality.
  arXiv  Detail & Related papers  (2026-07-06T15:53:09Z)
• Scene and Human in One World: Reconstruction in a Feedforward Pass [32.88487546112625]
  Show is a mask-promptable human mesh recovery framework that couples feed-forward 3D scene reconstruction with Human Mesh Recovery.<n> SHOW injects human semantics and scale priors from parametric human models into normalized point-map prediction.<n>It constrains human mesh estimation, encouraging spatially consistent human placement and improved human-scene alignment.
  arXiv  Detail & Related papers  (2026-06-26T05:08:55Z)
• TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos [13.556917488868661]
  TROPHIES aims to jointly estimate dynamic humans, static scenes, and camera poses in one global coordinate frame.<n>A global alignment and optimization module couples both branches by enforcing scale consistency, contact priors, and cross-view temporal coherence.<n>Experiments on EgoHuman and EgoExo4D demonstrate that TROPHIES consistently outperforms existing paradigms in both global fidelity and human-scene consistency.
  arXiv  Detail & Related papers  (2026-06-01T15:00:18Z)
• EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agents [85.77432303199176]
  We propose EmbodMocap, a portable and affordable data collection pipeline using two moving iPhones.<n>Our key idea is to jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes.<n>Based on the collected data, we empower three embodied AI tasks: monocular human-scene-reconstruction, where we fine-tune feedforward models that output metric-scale, world-space aligned humans and scenes; physics-based character animation, where we prove our data could be used to scale human-object interaction skills and scene-aware motion tracking; and robot motion control, where we train a humanoid robot via
  arXiv  Detail & Related papers  (2026-02-26T16:53:41Z)
• CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitives [65.89192712575797]
  We introduce CRISP, a method that recovers simulatable human motion and scene geometry from monocular video.<n>Our approach reduces motion tracking failure rates from 55.2% to 6.9% on human-centric video benchmarks.<n>This demonstrates CRISP's ability to generate physically-valid human motion and interaction environments at scale.
  arXiv  Detail & Related papers  (2025-12-16T18:59:50Z)
• Dynamic Avatar-Scene Rendering from Human-centric Context [75.95641456716373]
  We propose bf Separate-then-Map (StM) strategy to bridge separately defined and optimized models.<n>StM significantly outperforms existing state-of-the-art methods in both visual quality and rendering accuracy.
  arXiv  Detail & Related papers  (2025-11-13T17:39:06Z)
• Physics-based Human Pose Estimation from a Single Moving RGB Camera [47.50334809388003]
  MoviCam is the first non-synthetic dataset containing ground-truth camera trajectories.<n> PhysDynPose is a physics-based method that incorporates scene geometry and physical constraints.<n>Our method robustly estimates both human and camera poses in world coordinates.
  arXiv  Detail & Related papers  (2025-07-23T11:04:30Z)
• ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos [18.73641648585445]
  Recent neural rendering advances have enabled holistic human-scene reconstruction but require pre-calibrated camera and human poses.<n>We introduce a novel unified framework that simultaneously performs camera tracking, human pose estimation and human-scene reconstruction in an online fashion.<n>Specifically, we design a human deformation module to reconstruct the details and enhance generalizability to out-of-distribution poses faithfully.
  arXiv  Detail & Related papers  (2025-04-17T17:59:02Z)
• WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction [63.666166249633385]
  We present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis.<n>Our method achieves SOTA performance in producing photorealistic renderings from the given monocular video.
  arXiv  Detail & Related papers  (2025-02-03T04:43:41Z)
• Joint Optimization for 4D Human-Scene Reconstruction in the Wild [59.322951972876716]
  We propose JOSH, a novel optimization-based method for 4D human-scene reconstruction in the wild from monocular videos.<n>Experiment results show JOSH achieves better results on both global human motion estimation and dense scene reconstruction.<n>We further design a more efficient model, JOSH3R, and directly train it with pseudo-labels from web videos.
  arXiv  Detail & Related papers  (2025-01-04T01:53:51Z)
• Gravity-Aware Monocular 3D Human-Object Reconstruction [73.25185274561139]
  This paper proposes a new approach for joint markerless 3D human motion capture and object trajectory estimation from monocular RGB videos. We focus on scenes with objects partially observed during a free flight. In the experiments, our approach achieves state-of-the-art accuracy in 3D human motion capture on various metrics.
  arXiv  Detail & Related papers  (2021-08-19T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.