Compensating Spatiotemporally Inconsistent Observations for Online Dynamic 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2505.01235v1
• Date: Fri, 02 May 2025 12:50:24 GMT
• Title: Compensating Spatiotemporally Inconsistent Observations for Online Dynamic 3D Gaussian Splatting
• Abstract summary: This paper identifies that errors such as noise in real-world recordings affect temporal inconsistency in online reconstruction.<n>We propose a method that enhances temporal consistency in online reconstruction from observations with temporal inconsistency which is inevitable in cameras.
• Score: 9.592278104887036
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Online reconstruction of dynamic scenes is significant as it enables learning scenes from live-streaming video inputs, while existing offline dynamic reconstruction methods rely on recorded video inputs. However, previous online reconstruction approaches have primarily focused on efficiency and rendering quality, overlooking the temporal consistency of their results, which often contain noticeable artifacts in static regions. This paper identifies that errors such as noise in real-world recordings affect temporal inconsistency in online reconstruction. We propose a method that enhances temporal consistency in online reconstruction from observations with temporal inconsistency which is inevitable in cameras. We show that our method restores the ideal observation by subtracting the learned error. We demonstrate that applying our method to various baselines significantly enhances both temporal consistency and rendering quality across datasets. Code, video results, and checkpoints are available at https://bbangsik13.github.io/OR2.

Related papers

• Revisiting the Current Frame: Physical-Trace-Guided Network Output Correction for Video Restoration [20.573571456618936]
  We propose ANCHOR, a model-agnostic framework that revisits the low-quality current frame as a temporally aligned anchor for video restoration correction.<n> Experiments on High Dynamic Range video reconstruction and video deraining balances demonstrate consistent improvements across various state-of-the-art restoration models.
  arXiv  Detail & Related papers  (2026-08-10T09:20:51Z)
• GeoStream: Toward Precise Camera Controlled Streaming Video Generation [85.01823220801256]
  Existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories.<n>Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive.<n>We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation.
  arXiv  Detail & Related papers  (2026-06-13T07:18:04Z)
• VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors [79.88300861391114]
  VidSplat is a training-free generative reconstruction framework.<n>We tackle two key challenges that enable the effective integration of generation and reconstruction.<n>Experiments on widely used benchmarks demonstrate our superior performance in sparse-view scene reconstruction.
  arXiv  Detail & Related papers  (2026-05-12T02:20:31Z)
• RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction [22.98233376526841]
  Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images.<n>We propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction.
  arXiv  Detail & Related papers  (2026-03-21T01:04:31Z)
• DiTVR: Zero-Shot Diffusion Transformer for Video Restoration [48.97196894658511]
  DiTVR is a zero shot video restoration framework that couples a diffusion transformer with trajectory aware attention and a flow consistent sampler.<n>Our attention mechanism aligns tokens along optical flow trajectories, with particular emphasis on vital layers that exhibit the highest sensitivity to temporal dynamics.<n>The flow guided sampler injects data consistency only into low-frequency bands, preserving high frequency priors while accelerating cache.
  arXiv  Detail & Related papers  (2025-08-11T09:54:45Z)
• GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering [54.489285024494855]
  Video stabilization is pivotal for video processing, as it removes unwanted shakiness while preserving the original user motion intent.<n>Existing approaches, depending on the domain they operate, suffer from several issues that degrade the user experience.<n>We introduce textbfGaVS, a novel 3D-grounded approach that reformulates video stabilization as a temporally-consistent local reconstruction and rendering' paradigm.
  arXiv  Detail & Related papers  (2025-06-30T15:24:27Z)
• Large-scale visual SLAM for in-the-wild videos [28.58692815339531]
  We introduce a robust pipeline designed to improve 3D reconstruction from casual videos.<n>We build upon recent deep visual odometry methods but increase robustness in several ways.<n>We demonstrate large-scale contiguous 3D models from several online videos in various environments.
  arXiv  Detail & Related papers  (2025-04-29T07:37:51Z)
• Temporal-Consistent Video Restoration with Pre-trained Diffusion Models [51.47188802535954]
  Video restoration (VR) aims to recover high-quality videos from degraded ones.<n>Recent zero-shot VR methods using pre-trained diffusion models (DMs) suffer from approximation errors during reverse diffusion and insufficient temporal consistency.<n>We present a novel a Posterior Maximum (MAP) framework that directly parameterizes video frames in the seed space of DMs, eliminating approximation errors.
  arXiv  Detail & Related papers  (2025-03-19T03:41:56Z)
• SplatVoxel: History-Aware Novel View Streaming without Temporal Training [29.759664150610362]
  We study the problem of novel view streaming from sparse-view videos.<n>Existing novel view synthesis methods struggle with temporal coherence and visual fidelity.<n>We propose a hybrid splat-voxel feed-forward scene reconstruction approach.
  arXiv  Detail & Related papers  (2025-03-18T20:00:47Z)
• Collaborative Feedback Discriminative Propagation for Video Super-Resolution [66.61201445650323]
  Key success of video super-resolution (VSR) methods stems mainly from exploring spatial and temporal information. Inaccurate alignment usually leads to aligned features with significant artifacts. propagation modules only propagate the same timestep features forward or backward.
  arXiv  Detail & Related papers  (2024-04-06T22:08:20Z)
• Toward Accurate and Temporally Consistent Video Restoration from Raw Data [20.430231283171327]
  We present a new VJDD framework by consistent and accurate latent space propagation. The proposed losses can circumvent the error accumulation problem caused by inaccurate flow estimation. Experiments demonstrate the leading VJDD performance in term of restoration accuracy, perceptual quality and temporal consistency.
  arXiv  Detail & Related papers  (2023-12-25T12:38:03Z)
• Video Dynamics Prior: An Internal Learning Approach for Robust Video Enhancements [83.5820690348833]
  We present a framework for low-level vision tasks that does not require any external training data corpus. Our approach learns neural modules by optimizing over a corrupted sequence, leveraging the weights of the coherence-temporal test and statistics internal statistics.
  arXiv  Detail & Related papers  (2023-12-13T01:57:11Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)
• Progressive Temporal Feature Alignment Network for Video Inpainting [51.26380898255555]
  Video convolution aims to fill in-temporal "corrupted regions" with plausible content. Current methods achieve this goal through attention, flow-based warping, or 3D temporal convolution. We propose 'Progressive Temporal Feature Alignment Network', which progressively enriches features extracted from the current frame with the warped feature from neighbouring frames.
  arXiv  Detail & Related papers  (2021-04-08T04:50:33Z)
• SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion [86.77318031029404]
  We propose a framework that performs scene reconstruction and semantic scene completion jointly in an incremental and real-time manner. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model.
  arXiv  Detail & Related papers  (2020-10-26T15:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.