SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

• URL: http://arxiv.org/abs/2603.26481v1
• Date: Fri, 27 Mar 2026 14:44:52 GMT
• Title: SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras
• Abstract summary: High-quality dynamic scenes typically require dense arrays or even hundreds of synchronized cameras.<n>We propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations.<n>We develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs.
• Score: 13.258108161849727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. The reliance on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches.

Related papers

• OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields [69.80996430631079]
  We propose OmniX, a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion.<n>By leveraging the sparse and low-rank structure of 3D motion, these tokens generate trajectory fields for all pixels across all images while efficiently preserving global interactions.
  arXiv  Detail & Related papers  (2026-07-12T17:05:18Z)
• 4D Reconstruction from Sparse Dynamic Cameras [32.72043828725801]
  We focus on an alternative practical way, i.e. sparse dynamic camera setup, where a handful of cameras capture the same subjects.<n>This setup introduces multi-view constraints and remains practical for real-world video production such as sports, concerts, and TV shows.<n>We introduce LetCamsGo, a new real-world video dataset with 5 sequences across 4 diverse environments recorded by three independently moving cameras and one fixed camera.
  arXiv  Detail & Related papers  (2026-06-03T08:31:43Z)
• Syn4D: A Multiview Synthetic 4D Dataset [92.50132053229817]
  3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision.<n>To address this limitation, we introduce Syn4D, a multiview synthetic dynamic scenes.<n>Key feature of Syn4D is the ability to unproject any pixel into 3D.
  arXiv  Detail & Related papers  (2026-05-06T17:59:55Z)
• Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos [14.791378262503967]
  We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple cameras.<n>We propose a two-stage optimization framework that decouples the task robust camera tracking and dense depth.<n>Our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2026-03-12T15:31:32Z)
• MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer [45.19539316971492]
  MoRe is a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos.<n>Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure.<n>Experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.
  arXiv  Detail & Related papers  (2026-03-05T11:51:07Z)
• Dream4D: Lifting Camera-Controlled I2V towards Spatiotemporally Consistent 4D Generation [3.1852855132066673]
  Current approaches often struggle to maintain view consistency while handling complex scene dynamics.<n>This framework is the first to leverage both rich temporal priors video diffusion models and geometric awareness of the reconstruction models.<n>It significantly facilitates 4D generation and shows higher quality (e.g., mPSNR, mSSIM) over existing methods.
  arXiv  Detail & Related papers  (2025-08-11T08:55:47Z)
• 4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives [115.67081491747943]
  Dynamic 3D scene representation and novel view synthesis are crucial for enabling AR/VR and metaverse applications.<n>We reformulate the reconstruction of a time-varying 3D scene as approximating its underlying 4D volume.<n>We derive several compact variants that effectively reduce the memory footprint to address its storage bottleneck.
  arXiv  Detail & Related papers  (2024-12-30T05:30:26Z)
• Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera [49.82535393220003]
  Dyn-HaMR is the first approach to reconstruct 4D global hand motion from monocular videos recorded by dynamic cameras in the wild.<n>We show that our approach significantly outperforms state-of-the-art methods in terms of 4D global mesh recovery.<n>This establishes a new benchmark for hand motion reconstruction from monocular video with moving cameras.
  arXiv  Detail & Related papers  (2024-12-17T12:43:10Z)
• R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras [106.52409577316389]
  R3D3 is a multi-camera system for dense 3D reconstruction and ego-motion estimation. Our approach exploits spatial-temporal information from multiple cameras, and monocular depth refinement. We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments.
  arXiv  Detail & Related papers  (2023-08-28T17:13:49Z)
• NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields [99.57774680640581]
  We present an efficient framework capable of fast reconstruction, compact modeling, and streamable rendering. We propose to decompose the 4D space according to temporal characteristics. Points in the 4D space are associated with probabilities belonging to three categories: static, deforming, and new areas.
  arXiv  Detail & Related papers  (2022-10-28T07:11:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.