Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos

• URL: http://arxiv.org/abs/2603.12064v2
• Date: Sat, 14 Mar 2026 21:48:41 GMT
• Title: Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos
• Abstract summary: We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple cameras.<n>We propose a two-stage optimization framework that decouples the task robust camera tracking and dense depth.<n>Our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks.
• Score: 14.791378262503967
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple freely moving cameras -- a setting that arises naturally when multiple observers capture a shared event. Prior approaches either handle only single-camera input or require rigidly mounted, pre-calibrated camera rigs, limiting their practical applicability. We propose a two-stage optimization framework that decouples the task into robust camera tracking and dense depth refinement. In the first stage, we extend single-camera visual SLAM to the multi-camera setting by constructing a spatiotemporal connection graph that exploits both intra-camera temporal continuity and inter-camera spatial overlap, enabling consistent scale and robust tracking. To ensure robustness under limited overlap, we introduce a wide-baseline initialization strategy using feed-forward reconstruction models. In the second stage, we refine depth and camera poses by optimizing dense inter- and intra-camera consistency using wide-baseline optical flow. Additionally, we introduce MultiCamRobolab, a new real-world dataset with ground-truth poses from a motion capture system. Finally, we demonstrate that our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks, while requiring less memory.

Related papers

• 4D Reconstruction from Sparse Dynamic Cameras [32.72043828725801]
  We focus on an alternative practical way, i.e. sparse dynamic camera setup, where a handful of cameras capture the same subjects.<n>This setup introduces multi-view constraints and remains practical for real-world video production such as sports, concerts, and TV shows.<n>We introduce LetCamsGo, a new real-world video dataset with 5 sequences across 4 diverse environments recorded by three independently moving cameras and one fixed camera.
  arXiv  Detail & Related papers  (2026-06-03T08:31:43Z)
• SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras [13.258108161849727]
  High-quality dynamic scenes typically require dense arrays or even hundreds of synchronized cameras.<n>We propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations.<n>We develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs.
  arXiv  Detail & Related papers  (2026-03-27T14:44:52Z)
• MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects [19.26912938809402]
  Multi-camera dynamic Augmented Reality (AR) applications require a camera pose estimation to leverage individual information from each camera in one common system.<n>While commonly cameras are calibrated in an initial step or updated through the constant use of markers, another option is to leverage information already present in the scene, like known objects.<n>We propose a constant dynamic camera pose estimation leveraging FotemporalV overlaps of known objects on the fly.
  arXiv  Detail & Related papers  (2026-03-24T06:26:28Z)
• E-4DGS: High-Fidelity Dynamic Reconstruction from the Multi-view Event Cameras [44.78881339601845]
  Event cameras offer advantages of low power, high temporal resolution and high dynamic range.<n>New perspective to address the scene reconstruction challenges in high-speed motion.
  arXiv  Detail & Related papers  (2025-08-13T16:09:36Z)
• FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video [52.33896173943054]
  Egocentric motion capture with a head-mounted body-facing stereo camera is crucial for VR and AR applications.<n>Existing methods rely on synthetic pretraining and struggle to generate smooth and accurate predictions in real-world settings.<n>We propose FRAME, a simple yet effective architecture that combines device pose and camera feeds for state-of-the-art body pose prediction.
  arXiv  Detail & Related papers  (2025-03-29T14:26:06Z)
• BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splatting [8.480548767680894]
  We introduce BARD-GS, a novel approach for robust dynamic scene reconstruction that effectively handles blurry inputs and imprecise camera poses.<n>By explicitly decomposing motion blur into camera motion blur and object motion blur, we achieve significantly improved rendering results in dynamic regions.
  arXiv  Detail & Related papers  (2025-03-20T04:23:52Z)
• CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models [89.63787060844409]
  CameraCtrl II is a framework that enables large-scale dynamic scene exploration through a camera-controlled video diffusion model.<n>We take an approach that progressively expands the generation of dynamic scenes.
  arXiv  Detail & Related papers  (2025-03-13T17:42:01Z)
• Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera [49.82535393220003]
  Dyn-HaMR is the first approach to reconstruct 4D global hand motion from monocular videos recorded by dynamic cameras in the wild.<n>We show that our approach significantly outperforms state-of-the-art methods in terms of 4D global mesh recovery.<n>This establishes a new benchmark for hand motion reconstruction from monocular video with moving cameras.
  arXiv  Detail & Related papers  (2024-12-17T12:43:10Z)
• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• VICAN: Very Efficient Calibration Algorithm for Large Camera Networks [49.17165360280794]
  We introduce a novel methodology that extends Pose Graph Optimization techniques. We consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step. Our framework retains compatibility with traditional PGO solvers, but its efficacy benefits from a custom-tailored optimization scheme.
  arXiv  Detail & Related papers  (2024-03-25T17:47:03Z)
• Event-based Continuous Color Video Decompression from Single Frames [36.4263932473053]
  We present ContinuityCam, a novel approach to generate a continuous video from a single static RGB image and an event camera stream.<n>Our approach combines continuous long-range motion modeling with a neural synthesis model, enabling frame prediction at arbitrary times within the events.
  arXiv  Detail & Related papers  (2023-11-30T18:59:23Z)
• Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time [101.91824315554682]
  In this work, we aim ambitiously for a more realistic and challenging task - joint video multi-frame and deblurring under unknown exposure time. We first adopt a variant of supervised contrastive learning to construct an exposure-aware representation from input blurred frames. We then build our video reconstruction network upon the exposure and motion representation by progressive exposure-adaptive convolution and motion refinement.
  arXiv  Detail & Related papers  (2023-03-27T09:43:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.