READ: Large-Scale Neural Scene Rendering for Autonomous Driving
- URL: http://arxiv.org/abs/2205.05509v1
- Date: Wed, 11 May 2022 14:02:14 GMT
- Title: READ: Large-Scale Neural Scene Rendering for Autonomous Driving
- Authors: Zhuopeng Li, Lu Li, Zeyu Ma, Ping Zhang, Junbo Chen, Jianke Zhu
- Abstract summary: A large-scale neural rendering method is proposed to synthesize the autonomous driving scene.
Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes.
- Score: 21.144110676687667
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Synthesizing free-view photo-realistic images is an important task in
multimedia. With the development of advanced driver assistance systems~(ADAS)
and their applications in autonomous vehicles, experimenting with different
scenarios becomes a challenge. Although the photo-realistic street scenes can
be synthesized by image-to-image translation methods, which cannot produce
coherent scenes due to the lack of 3D information. In this paper, a large-scale
neural rendering method is proposed to synthesize the autonomous driving
scene~(READ), which makes it possible to synthesize large-scale driving
scenarios on a PC through a variety of sampling schemes. In order to represent
driving scenarios, we propose an {\omega} rendering network to learn neural
descriptors from sparse point clouds. Our model can not only synthesize
realistic driving scenes but also stitch and edit driving scenes. Experiments
show that our model performs well in large-scale driving scenarios.
Related papers
- AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction [17.600027937450342]
AutoSplat is a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes.
Our method enables multi-view consistent simulation of challenging scenarios including lane changes.
arXiv Detail & Related papers (2024-07-02T18:36:50Z) - Urban Scene Diffusion through Semantic Occupancy Map [49.20779809250597]
UrbanDiffusion is a 3D diffusion model conditioned on a Bird's-Eye View (BEV) map.
Our model learns the data distribution of scene-level structures within a latent space.
After training on real-world driving datasets, our model can generate a wide range of diverse urban scenes.
arXiv Detail & Related papers (2024-03-18T11:54:35Z) - Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents [49.11254369459406]
ChatSim is the first system that enables editable photo-realistic 3D driving scene simulations via natural language commands with external digital assets.
ChatSim employs a novel multi-camera neural field method to generate photo-realistic outcomes.
arXiv Detail & Related papers (2024-02-08T15:26:28Z) - Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting [32.59889755381453]
Recent methods extend NeRF by incorporating tracked vehicle poses to animate vehicles, enabling photo-realistic view of dynamic urban street scenes.
We introduce Street Gaussians, a new explicit scene representation that tackles these limitations.
The proposed method consistently outperforms state-of-the-art methods across all datasets.
arXiv Detail & Related papers (2024-01-02T18:59:55Z) - Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs [83.9783063609389]
We propose a method based on domain-invariant scene representation to directly synthesize traffic scene imagery without rendering.
Specifically, we rely on synthetic scene graphs as our internal representation and introduce an unsupervised neural network architecture for realistic traffic scene synthesis.
arXiv Detail & Related papers (2023-03-15T09:26:29Z) - Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
Bird's-Eye View (BEV) Perception has received increasing attention in recent years.
Data-driven simulation for autonomous driving has been a focal point of recent research.
We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
arXiv Detail & Related papers (2023-01-11T18:39:34Z) - SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions.
We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
arXiv Detail & Related papers (2021-01-16T22:51:43Z) - Photorealism in Driving Simulations: Blending Generative Adversarial
Image Synthesis with Rendering [0.0]
We introduce a hybrid generative neural graphics pipeline for improving the visual fidelity of driving simulations.
We form 2D semantic images from 3D scenery consisting of simple object models without textures.
These semantic images are then converted into photorealistic RGB images with a state-of-the-art Generative Adrial Network (GAN) trained on real-world driving scenes.
arXiv Detail & Related papers (2020-07-31T03:25:17Z) - LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World [84.57894492587053]
We develop a novel simulator that captures both the power of physics-based and learning-based simulation.
We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation.
We showcase LiDARsim's usefulness for perception algorithms-testing on long-tail events and end-to-end closed-loop evaluation on safety-critical scenarios.
arXiv Detail & Related papers (2020-06-16T17:44:35Z) - SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving [27.948417322786575]
We present a simple yet effective approach to generate realistic scenario sensor data.
Our approach uses texture-mapped surfels to efficiently reconstruct the scene from an initial vehicle pass or set of passes.
We then leverage a SurfelGAN network to reconstruct realistic camera images for novel positions and orientations of the self-driving vehicle.
arXiv Detail & Related papers (2020-05-08T04:01:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.