AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

• URL: http://arxiv.org/abs/2407.02598v2
• Date: Thu, 4 Jul 2024 02:18:54 GMT
• Title: AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction
• Authors: Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tongtong Cao, Dongfeng Bai, Yuan Ren, Bingbing Liu,
• Abstract summary: AutoSplat is a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. Our method enables multi-view consistent simulation of challenging scenarios including lane changes.
• Score: 17.600027937450342
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Realistic scene reconstruction and view synthesis are essential for advancing autonomous driving systems by simulating safety-critical scenarios. 3D Gaussian Splatting excels in real-time rendering and static scene reconstructions but struggles with modeling driving scenarios due to complex backgrounds, dynamic objects, and sparse views. We propose AutoSplat, a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, our method enables multi-view consistent simulation of challenging scenarios including lane changes. Leveraging 3D templates, we introduce a reflected Gaussian consistency constraint to supervise both the visible and unseen side of foreground objects. Moreover, to model the dynamic appearance of foreground objects, we estimate residual spherical harmonics for each foreground Gaussian. Extensive experiments on Pandaset and KITTI demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. Visit our project page at https://autosplat.github.io/.

Related papers

• OmniRe: Omni Urban Scene Reconstruction [78.99262488964423]
  We introduce OmniRe, a holistic approach for efficiently reconstructing high-fidelity dynamic urban scenes from on-device logs. We propose a comprehensive 3DGS framework for driving scenes, named OmniRe, that allows for accurate, full-length reconstruction of diverse dynamic objects in a driving log.
  arXiv  Detail & Related papers  (2024-08-29T17:56:33Z)
• Solving Motion Planning Tasks with a Scalable Generative Model [15.858076912795621]
  We present an efficient solution based on generative models which learns the dynamics of the driving scenes. Our innovative design allows the model to operate in both full-Autoregressive and partial-Autoregressive modes. We conclude that the proposed generative model may serve as a foundation for a variety of motion planning tasks.
  arXiv  Detail & Related papers  (2024-07-03T03:57:05Z)
• Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata [70.9375320609781]
  We aim to generate fine-grained 3D geometry from large-scale sparse LiDAR scans, abundantly captured by autonomous vehicles (AV) We propose hierarchical Generative Cellular Automata (hGCA), a spatially scalable 3D generative model, which grows geometry with local kernels following, in a coarse-to-fine manner, equipped with a light-weight planner to induce global consistency.
  arXiv  Detail & Related papers  (2024-06-12T14:56:56Z)
• Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting [32.59889755381453]
  Recent methods extend NeRF by incorporating tracked vehicle poses to animate vehicles, enabling photo-realistic view of dynamic urban street scenes. We introduce Street Gaussians, a new explicit scene representation that tackles these limitations. The proposed method consistently outperforms state-of-the-art methods across all datasets.
  arXiv  Detail & Related papers  (2024-01-02T18:59:55Z)
• DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes [57.12439406121721]
  We present DrivingGaussian, an efficient and effective framework for surrounding dynamic autonomous driving scenes. For complex scenes with moving objects, we first sequentially and progressively model the static background of the entire scene. We then leverage a composite dynamic Gaussian graph to handle multiple moving objects. We further use a LiDAR prior for Gaussian Splatting to reconstruct scenes with greater details and maintain panoramic consistency.
  arXiv  Detail & Related papers  (2023-12-13T06:30:51Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)
• LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World [84.57894492587053]
  We develop a novel simulator that captures both the power of physics-based and learning-based simulation. We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation. We showcase LiDARsim's usefulness for perception algorithms-testing on long-tail events and end-to-end closed-loop evaluation on safety-critical scenarios.
  arXiv  Detail & Related papers  (2020-06-16T17:44:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.