DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos

• URL: http://arxiv.org/abs/2405.17705v3
• Date: Tue, 05 Nov 2024 18:02:53 GMT
• Title: DC-Gaussian: Improving 3D Gaussian Splatting for Reflective Dash Cam Videos
• Abstract summary: We present DC-Gaussian, a new method for generating novel views from in-vehicle dash cam videos. Our method achieves state-of-the-art performance in novel view synthesis.
• Score: 23.967885144420613
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DC-Gaussian, a new method for generating novel views from in-vehicle dash cam videos. While neural rendering techniques have made significant strides in driving scenarios, existing methods are primarily designed for videos collected by autonomous vehicles. However, these videos are limited in both quantity and diversity compared to dash cam videos, which are more widely used across various types of vehicles and capture a broader range of scenarios. Dash cam videos often suffer from severe obstructions such as reflections and occlusions on the windshields, which significantly impede the application of neural rendering techniques. To address this challenge, we develop DC-Gaussian based on the recent real-time neural rendering technique 3D Gaussian Splatting (3DGS). Our approach includes an adaptive image decomposition module to model reflections and occlusions in a unified manner. Additionally, we introduce illumination-aware obstruction modeling to manage reflections and occlusions under varying lighting conditions. Lastly, we employ a geometry-guided Gaussian enhancement strategy to improve rendering details by incorporating additional geometry priors. Experiments on self-captured and public dash cam videos show that our method not only achieves state-of-the-art performance in novel view synthesis, but also accurately reconstructing captured scenes getting rid of obstructions. See the project page for code, data: https://linhanwang.github.io/dcgaussian/.

Related papers

• Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis [53.67632483195509]
  We introduce StreetNVS, a video diffusion framework that conditions on all three signals through a Reference-Enhanced Camera Attention module.<n>We show capabilities of synthesizing coherent videos along extreme out-of-trajectory paths such as elevation, lane-shift, pullback, and rotation.
  arXiv  Detail & Related papers  (2026-06-01T02:37:56Z)
• Pixel-to-4D: Camera-Controlled Image-to-Video Generation with Dynamic 3D Gaussians [7.051077403685518]
  Humans excel at forecasting the future dynamics of a scene given just a single image.<n>Video generation models that can mimic this ability are an essential component for intelligent systems.<n>Recent approaches have improved temporal coherence and 3D consistency in single-image-conditioned video generation.
  arXiv  Detail & Related papers  (2026-01-02T13:04:47Z)
• DriveExplorer: Images-Only Decoupled 4D Reconstruction with Progressive Restoration for Driving View Extrapolation [12.714087160353317]
  This paper presents an effective solution for view extrapolation in autonomous driving scenarios.<n>Recent approaches focus on generating shifted novel view images from given viewpoints using diffusion models.<n>Compared with baselines, our method produces higher-quality images at novel extrapolated viewpoints.
  arXiv  Detail & Related papers  (2025-12-30T04:41:56Z)
• S^2VG: 3D Stereoscopic and Spatial Video Generation via Denoising Frame Matrix [60.060882467801484]
  We present a pose-free and training-free method that leverages an off-the-shelf monocular video generation model to produce immersive 3D videos.<n>Our approach first warps the generated monocular video into pre-defined camera viewpoints using estimated depth information, then applies a novel textitframe matrix inpainting framework.<n>We validate the efficacy of our proposed method by conducting experiments on videos from various generative models, such as Sora, Lumiere, WALT, and Zeroscope.
  arXiv  Detail & Related papers  (2025-08-11T14:50:03Z)
• Direct Motion Models for Assessing Generated Videos [38.04485796547767]
  A current limitation of video generative video models is that they generate plausible looking frames, but poor motion.<n>Here we go beyond FVD by developing a metric which better measures plausible object interactions and motion.<n>We show that using point tracks instead of pixel reconstruction or action recognition features results in a metric which is markedly more sensitive to temporal distortions in synthetic data.
  arXiv  Detail & Related papers  (2025-04-30T22:34:52Z)
• NovelGS: Consistent Novel-view Denoising via Large Gaussian Reconstruction Model [57.92709692193132]
  NovelGS is a diffusion model for Gaussian Splatting given sparse-view images. We leverage the novel view denoising through a transformer-based network to generate 3D Gaussians.
  arXiv  Detail & Related papers  (2024-11-25T07:57:17Z)
• EMD: Explicit Motion Modeling for High-Quality Street Gaussian Splatting [22.590036750925627]
  Photorealistic reconstruction of street scenes is essential for developing real-world simulators in autonomous driving. Recent methods based on 3D/4D Gaussian Splatting (GS) have demonstrated promising results, but they still encounter challenges in complex street scenes due to the unpredictable motion of dynamic objects. We propose Explicit Motion Decomposition (EMD), which models the motions of dynamic objects by introducing learnable motion embeddings to the Gaussians.
  arXiv  Detail & Related papers  (2024-11-23T15:10:04Z)
• UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations [23.346765633086452]
  How to effectively simulate fisheye cameras in driving scene remains an unsolved problem. We propose UniGaussian, a novel approach that learns a unified 3D Gaussian representation from multiple camera models. Our approach models multiple sensors (pinhole and fisheye cameras) and modalities (depth, semantic, normal and LiDAR point clouds)
  arXiv  Detail & Related papers  (2024-11-22T21:59:46Z)
• SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix [60.48666051245761]
  We propose a pose-free and training-free approach for generating 3D stereoscopic videos. Our method warps a generated monocular video into camera views on stereoscopic baseline using estimated video depth. We develop a disocclusion boundary re-injection scheme that further improves the quality of video inpainting.
  arXiv  Detail & Related papers  (2024-06-29T08:33:55Z)
• Splatter a Video: Video Gaussian Representation for Versatile Processing [48.9887736125712]
  Video representation is crucial for various down-stream tasks, such as tracking,depth prediction,segmentation,view synthesis,and editing. We introduce a novel explicit 3D representation-video Gaussian representation -- that embeds a video into 3D Gaussians. It has been proven effective in numerous video processing tasks, including tracking, consistent video depth and feature refinement, motion and appearance editing, and stereoscopic video generation.
  arXiv  Detail & Related papers  (2024-06-19T22:20:03Z)
• LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes [73.65115834242866]
  Photorealistic simulation plays a crucial role in applications such as autonomous driving. However, reconstruction quality suffers on street scenes due to collinear camera motions and sparser samplings at higher speeds. We propose several insights that allow a better utilization of Lidar data to improve NeRF quality on street scenes.
  arXiv  Detail & Related papers  (2024-05-01T23:07:12Z)
• Bootstrap 3D Reconstructed Scenes from 3D Gaussian Splatting [10.06208115191838]
  We present a bootstrapping method to enhance the rendering of novel views using trained 3D-GS. Our results indicate that bootstrapping effectively reduces artifacts, as well as clear enhancements on the evaluation metrics.
  arXiv  Detail & Related papers  (2024-04-29T12:57:05Z)
• SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior [53.52396082006044]
  Current methods struggle to maintain rendering quality at the viewpoint that deviates significantly from the training viewpoints. This issue stems from the sparse training views captured by a fixed camera on a moving vehicle. We propose a novel approach that enhances the capacity of 3DGS by leveraging prior from a Diffusion Model.
  arXiv  Detail & Related papers  (2024-03-29T09:20:29Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.