LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstruction

• URL: http://arxiv.org/abs/2512.13680v1
• Date: Mon, 15 Dec 2025 18:59:04 GMT
• Title: LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstruction
• Abstract summary: LASER is a training-free framework that converts an offline reconstruction model into a streaming system.<n>We introduce layer-wise scale alignment, which segments depth predictions into discrete layers.<n>Experiments show that LASER achieves state-of-the-art performance on camera pose estimation and point map reconstruction %quality with offline models.
• Score: 28.14207161940146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent feed-forward reconstruction models like VGGT and $π^3$ achieve impressive reconstruction quality but cannot process streaming videos due to quadratic memory complexity, limiting their practical deployment. While existing streaming methods address this through learned memory mechanisms or causal attention, they require extensive retraining and may not fully leverage the strong geometric priors of state-of-the-art offline models. We propose LASER, a training-free framework that converts an offline reconstruction model into a streaming system by aligning predictions across consecutive temporal windows. We observe that simple similarity transformation ($\mathrm{Sim}(3)$) alignment fails due to layer depth misalignment: monocular scale ambiguity causes relative depth scales of different scene layers to vary inconsistently between windows. To address this, we introduce layer-wise scale alignment, which segments depth predictions into discrete layers, computes per-layer scale factors, and propagates them across both adjacent windows and timestamps. Extensive experiments show that LASER achieves state-of-the-art performance on camera pose estimation and point map reconstruction %quality with offline models while operating at 14 FPS with 6 GB peak memory on a RTX A6000 GPU, enabling practical deployment for kilometer-scale streaming videos. Project website: $\href{https://neu-vi.github.io/LASER/}{\texttt{https://neu-vi.github.io/LASER/}}$

Related papers

• Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors [54.51965288351254]
  We present an online 3D vision-language model that enables real-time spatial understanding from streaming video.<n>Our approach adopts an autoregressive streaming control modeling based on the LLM's next-token prediction to learn when to respond.<n>To address the scarcity of streaming 3D-language data, we further develop a scalable data generation pipeline.
  arXiv  Detail & Related papers  (2026-06-05T04:16:24Z)
• S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction [57.07346645250984]
  Streaming Semantic Gaussian Splatting (S2GS) is a strictly causal, incremental 3D Gaussian semantic field framework.<n>It does not leverage future frames and continuously updates scene geometry, appearance, and instance-level semantics without reprocessing historical frames.<n>S2GS matches or outperforms strong offline baselines on joint reconstruction-and-understanding benchmarks.
  arXiv  Detail & Related papers  (2026-03-15T05:48:55Z)
• MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAE [95.16927402591227]
  MotionCrafter is a video diffusion-based framework that jointly reconstructs 4D geometry and estimates dense motion from a monocular video.<n>We show that MotionCrafter achieves state-of-the-art performance in both geometry reconstruction and dense scene flow estimation.
  arXiv  Detail & Related papers  (2026-02-09T17:58:12Z)
• LaVR: Scene Latent Conditioned Generative Video Trajectory Re-Rendering using Large 4D Reconstruction Models [52.656349227001925]
  Given a monocular video, the goal of video re-rendering is to generate views of the scene from a novel camera trajectory.<n>Existing methods face two distinct challenges.<n>We propose to address these challenges by using the implicit geometric knowledge embedded in the latent space of a large 4D reconstruction model.
  arXiv  Detail & Related papers  (2026-01-21T05:46:03Z)
• STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer [72.88105562624838]
  We present STream3R, a novel approach to 3D reconstruction that reformulates pointmap prediction as a decoder-only Transformer problem.<n>By learning geometric priors from large-scale 3D datasets, STream3R generalizes well to diverse and challenging scenarios.<n>Our results underscore the potential of causal Transformer models for online 3D perception, paving the way for real-time 3D understanding in streaming environments.
  arXiv  Detail & Related papers  (2025-08-14T17:58:05Z)
• A LoD of Gaussians: Unified Training and Rendering for Ultra-Large Scale Reconstruction with External Memory [14.128827855029636]
  We introduce A LoD of Gaussians, a framework for training and rendering ultra-large-scale scenes on a single consumer-grade GPU.<n>A hybrid data structure combining Gaussian hierarchies with Sequential Point Trees enables efficient, view-dependent LoD selection.<n>A lightweight caching and view scheduling system exploits temporal coherence to support real-time streaming and rendering.
  arXiv  Detail & Related papers  (2025-07-01T18:12:43Z)
• Minute-Long Videos with Dual Parallelisms [57.22737565366549]
  Diffusion Transformer (DiT)-based video diffusion models generate high-quality videos at scale but incur prohibitive processing latency and memory costs for long videos.<n>We propose a novel distributed inference strategy, termed DualParal.<n>Instead of generating an entire video on a single GPU, we parallelize both temporal frames and model layers across GPUs.
  arXiv  Detail & Related papers  (2025-05-27T11:55:22Z)
• VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [54.66217340264935]
  VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis.<n>It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
  arXiv  Detail & Related papers  (2025-01-03T18:52:36Z)
• Pyramidal Flow Matching for Efficient Video Generative Modeling [67.03504440964564]
  This work introduces a unified pyramidal flow matching algorithm.<n>It sacrifices the original denoising trajectory as a series of pyramid stages, where only the final stage operates at the full resolution.<n>The entire framework can be optimized in an end-to-end manner and with a single unified Diffusion Transformer (DiT)
  arXiv  Detail & Related papers  (2024-10-08T12:10:37Z)
• SIGMA:Sinkhorn-Guided Masked Video Modeling [69.31715194419091]
  Sinkhorn-guided Masked Video Modelling ( SIGMA) is a novel video pretraining method. We distribute features of space-time tubes evenly across a limited number of learnable clusters. Experimental results on ten datasets validate the effectiveness of SIGMA in learning more performant, temporally-aware, and robust video representations.
  arXiv  Detail & Related papers  (2024-07-22T08:04:09Z)
• Spatial-Temporal Deep Embedding for Vehicle Trajectory Reconstruction from High-Angle Video [1.8520147498637294]
  We develop a model that imposes parity constraints at both pixel and instance levels to generate instance-aware embeddings for vehicle segmentation on STMap. The designed model is applied to process all public NGSIM US-101 videos to generate complete vehicle trajectories.
  arXiv  Detail & Related papers  (2022-09-17T22:32:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.