UniFusion: Sparse-View 4D Reconstruction via Unified Spatio-temporal Depth Alignment

• URL: http://arxiv.org/abs/2609.05888v1
• Date: Sat, 05 Sep 2026 05:19:00 GMT
• Title: UniFusion: Sparse-View 4D Reconstruction via Unified Spatio-temporal Depth Alignment
• Abstract summary: A key challenge arises from limited cross-view overlap and temporal variation, making monocular depth predictions inconsistent across views time.<n>Existing methods align spatial and temporal dimensions in separate stages, requiring segmentation masks.<n>We propose a unified spatial-temporal depth alignment framework that resolves cross-view and cross-time inconsistencies without distinguishing/background.
• Score: 14.151508266012463
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we address the challenging problem of 4D reconstruction from sparse-view videos. This setup usually relies on monocular depth estimation to provide priors for the reconstruction model. A key challenge arises from limited cross-view overlap and temporal variation, making monocular depth predictions inconsistent across views and time. Existing methods align spatial and temporal dimensions in separate stages, requiring foreground segmentation masks while failing to leverage temporal cues for cross-view alignment. Contrary to these methods, we propose a unified spatial-temporal depth alignment framework that jointly resolves cross-view and cross-time inconsistencies without distinguishing foreground/background. Our method represents depth maps across views and time as a set of spatio-temporal neural fields. This representation not only yields fast convergence, but also captures spatio-temporal correlation among depth maps implicitly, without dependence on external segmentation/tracking models. We also propose a multi-view depth-order loss while leveraging the classic scale-and-shift-invariant loss to further improve the final depth quality. The aligned depths initialize and supervise Gaussian splatting models for 4D reconstruction. Experiments on Ego-Exo4D and EgoHuman demonstrate that our improved depth alignment substantially benefits dynamic Gaussian-splatting-based reconstruction methods for novel-time/view synthesis and geometry accuracy/consistency.

Related papers

• Ground4D: Spatially-Grounded Feedforward 4D Reconstruction for Unstructured Off-Road Scenes [15.758026389335669]
  Ground4D is a spatially-grounded 4D feedforward framework for pose-free off-road reconstruction.<n>In experiments on ORAD-3D and RELLIS-3D, Ground4D consistently outperforms existing feedforward methods in reconstruction quality.
  arXiv  Detail & Related papers  (2026-05-06T02:58:47Z)
• Face Anything: 4D Face Reconstruction from Any Image Sequence [49.395407357499074]
  We present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction.<n>By jointly predicting depth and canonical coordinates, our method enables accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking.
  arXiv  Detail & Related papers  (2026-04-21T17:22:39Z)
• Stitch4D: Sparse Multi-Location 4D Urban Reconstruction via Spatio-Temporal Interpolation [5.878115656035432]
  We propose a unified 4D reconstruction framework that explicitly compensates for missing spatial coverage in sparse observations.<n>By restoring intermediate coverage before optimization, Stitch4D prevents geometric collapse and reconstructs coherent geometry and smooth scene dynamics.<n> Experimental results on U-S4D show that Stitch4D surpasses representative 4D reconstruction baselines and achieves superior visual quality.
  arXiv  Detail & Related papers  (2026-04-09T07:45:51Z)
• \textit{4DSurf}: High-Fidelity Dynamic Scene Surface Reconstruction [53.82595920033838]
  This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS)<n>We propose textit4DSurf'', a novel and unified framework for generic dynamic surface reconstruction.<n>Our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance.
  arXiv  Detail & Related papers  (2026-03-30T06:09:37Z)
• Seurat: From Moving Points to Depth [66.65189052568209]
  We propose a novel method that infers relative depth by examining the spatial relationships and temporal evolution of a set of tracked 2D trajectories.<n>Our approach achieves temporally smooth, high-accuracy depth predictions across diverse domains.
  arXiv  Detail & Related papers  (2025-04-20T17:37:02Z)
• DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generation [50.01520547454224]
  Current generative models struggle to synthesize 4D driving scenes that simultaneously support temporal extrapolation and spatial novel view synthesis (NVS)<n>We propose DiST-4D, which disentangles the problem into two diffusion processes: DiST-T, which predicts future metric depth and multi-view RGB sequences directly from past observations, and DiST-S, which enables spatial NVS by training only on existing viewpoints while enforcing cycle consistency.<n>Experiments demonstrate that DiST-4D achieves state-of-the-art performance in both temporal prediction and NVS tasks, while also delivering competitive performance in planning-related evaluations.
  arXiv  Detail & Related papers  (2025-03-19T13:49:48Z)
• Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking [52.393359791978035]
  Motion2VecSets is a 4D diffusion model for dynamic surface reconstruction from point cloud sequences. We parameterize 4D dynamics with latent sets instead of using global latent codes. For more temporally-coherent object tracking, we synchronously denoise deformation latent sets and exchange information across multiple frames.
  arXiv  Detail & Related papers  (2024-01-12T15:05:08Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• Learning non-rigid surface reconstruction from spatio-temporal image patches [0.0]
  We present a method to reconstruct a dense-temporal depth map of a deformable object from a video sequence. The estimation of depth is performed locally on non-temporal patches of the video, and the full depth video of entire shape is recovered by combining them together. We tested our method on both synthetic and Kinect data and experimentally observed that the reconstruction error is significantly lower than the one obtained using other approaches like conventional non-rigid structure.
  arXiv  Detail & Related papers  (2020-06-18T20:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.