VS3R: Robust Full-frame Video Stabilization via Deep 3D Reconstruction

• URL: http://arxiv.org/abs/2603.05851v1
• Date: Fri, 06 Mar 2026 03:15:06 GMT
• Title: VS3R: Robust Full-frame Video Stabilization via Deep 3D Reconstruction
• Abstract summary: We propose VS3R, a framework that synergizes feed-forward 3D reconstruction with generative video diffusion.<n>Our pipeline jointly estimates camera parameters, depth, and masks to ensure all-scenario reliability.<n>A Dual-Stream Video Diffusion Model restores disoccluded regions and rectifies artifacts by synergizing structural guidance with semantic anchors.
• Score: 8.43362854806854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video stabilization aims to mitigate camera shake but faces a fundamental trade-off between geometric robustness and full-frame consistency. While 2D methods suffer from aggressive cropping, 3D techniques are often undermined by fragile optimization pipelines that fail under extreme motions. To bridge this gap, we propose VS3R, a framework that synergizes feed-forward 3D reconstruction with generative video diffusion. Our pipeline jointly estimates camera parameters, depth, and masks to ensure all-scenario reliability, and introduces a Hybrid Stabilized Rendering module that fuses semantic and geometric cues for dynamic consistency. Finally, a Dual-Stream Video Diffusion Model restores disoccluded regions and rectifies artifacts by synergizing structural guidance with semantic anchors. Collectively, VS3R achieves high-fidelity, full-frame stabilization across diverse camera models and significantly outperforms state-of-the-art methods in robustness and visual quality.

Related papers

• 3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation [21.287558280133076]
  We introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem.<n>Our approach is driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency.<n>Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.
  arXiv  Detail & Related papers  (2026-06-14T08:56:45Z)
• I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation [56.33710337846449]
  I3DM is a novel implicit 3D-aware memory mechanism for consistent video scene generation.<n>At the core of our approach is a 3D-aware memory retrieval strategy.<n>To fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module.
  arXiv  Detail & Related papers  (2026-03-24T16:45:40Z)
• TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos [65.99602532894241]
  We introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry.<n>We also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting.<n>Results demonstrate that our method outperforms existing approaches in both video consistency and final reconstruction quality.
  arXiv  Detail & Related papers  (2026-03-18T14:02:09Z)
• Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery [60.51998732898099]
  We introduce HMRMamba, a new paradigm for 3D Human Mesh Recovery.<n>It pioneers the use of Structured State Space Models for their efficiency and long-range modeling prowess.<n>Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture.
  arXiv  Detail & Related papers  (2026-01-29T08:05:02Z)
• SPARK: Scalable Real-Time Point Cloud Aggregation with Multi-View Self-Calibration [1.3537117504260623]
  SPARK is a self-calibrating real-time multi-camera point cloud reconstruction framework.<n>It produces stable point clouds in dynamic scenes while scaling linearly with the number of cameras.
  arXiv  Detail & Related papers  (2026-01-13T10:32:22Z)
• GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering [54.489285024494855]
  Video stabilization is pivotal for video processing, as it removes unwanted shakiness while preserving the original user motion intent.<n>Existing approaches, depending on the domain they operate, suffer from several issues that degrade the user experience.<n>We introduce textbfGaVS, a novel 3D-grounded approach that reformulates video stabilization as a temporally-consistent local reconstruction and rendering' paradigm.
  arXiv  Detail & Related papers  (2025-06-30T15:24:27Z)
• FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF [77.94545888842883]
  This paper proposes a novel face video editing architecture built upon the dynamic face GAN-NeRF structure. By editing the latent code, multi-view consistent editing on the face can be ensured, as validated by multiview stereo reconstruction. We propose a stabilizer that maintains temporal coherence by preserving smooth changes of face expressions in consecutive frames.
  arXiv  Detail & Related papers  (2024-01-05T03:23:38Z)
• Neural Re-rendering for Full-frame Video Stabilization [144.9918806873405]
  We present an algorithm for full-frame video stabilization by first estimating dense warp fields. Full-frame stabilized frames can then be synthesized by fusing warped contents from neighboring frames.
  arXiv  Detail & Related papers  (2021-02-11T18:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.