GSAVS: Gaussian Splatting-based Autonomous Vehicle Simulator

• URL: http://arxiv.org/abs/2412.18816v1
• Date: Wed, 25 Dec 2024 07:52:09 GMT
• Title: GSAVS: Gaussian Splatting-based Autonomous Vehicle Simulator
• Authors: Rami Wilson,
• Abstract summary: We introduce GSAVS, an autonomous vehicle simulator that supports the creation and development of autonomous vehicle models. Every asset within the simulator is a 3D Gaussian splat, including the vehicles and the environment. The simulator runs within a classical 3D engine, rendering 3D Gaussian splats in real-time.
• Score: 0.0
• License:
• Abstract: Modern autonomous vehicle simulators feature an ever-growing library of assets, including vehicles, buildings, roads, pedestrians, and more. While this level of customization proves beneficial when creating virtual urban environments, this process becomes cumbersome when intending to train within a digital twin or a duplicate of a real scene. Gaussian splatting emerged as a powerful technique in scene reconstruction and novel view synthesis, boasting high fidelity and rendering speeds. In this paper, we introduce GSAVS, an autonomous vehicle simulator that supports the creation and development of autonomous vehicle models. Every asset within the simulator is a 3D Gaussian splat, including the vehicles and the environment. However, the simulator runs within a classical 3D engine, rendering 3D Gaussian splats in real-time. This allows the simulator to utilize the photorealism that 3D Gaussian splatting boasts while providing the customization and ease of use of a classical 3D engine.

Related papers

• HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving [48.84595398410479]
  HUGSIM is a closed-loop, photo-realistic, and real-time simulator for evaluating autonomous driving algorithms. We tackle challenges of novel view synthesis in closed-loop scenarios, including viewpoint extrapolation and 360-degree vehicle rendering. HUGSIM offers a comprehensive benchmark across more than 70 sequences from KITTI-360, nuScenes, and PandaSet, along with over 400 varying scenarios.
  arXiv  Detail & Related papers  (2024-12-02T17:07:59Z)
• AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction [17.600027937450342]
  AutoSplat is a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. Our method enables multi-view consistent simulation of challenging scenarios including lane changes.
  arXiv  Detail & Related papers  (2024-07-02T18:36:50Z)
• Deep Reinforcement Learning for Adverse Garage Scenario Generation [5.482809279542029]
  This thesis proposes an automated program generation framework for autonomous driving simulation testing. Based on deep reinforcement learning, this framework can generate different 2D ground script codes, on which 3D model files and map model files are built. The generated 3D ground scenes are displayed in the Carla simulator, where experimenters can use this scene for navigation algorithm simulation testing.
  arXiv  Detail & Related papers  (2024-07-01T14:41:18Z)
• $\textit{S}^3$Gaussian: Self-Supervised Street Gaussians for Autonomous Driving [82.82048452755394]
  Photorealistic 3D reconstruction of street scenes is a critical technique for developing real-world simulators for autonomous driving. Most existing street 3DGS methods require tracked 3D vehicle bounding boxes to decompose the static and dynamic elements. We propose a self-supervised street Gaussian ($textitS3$Gaussian) method to decompose dynamic and static elements from 4D consistency.
  arXiv  Detail & Related papers  (2024-05-30T17:57:08Z)
• Gaussian Shell Maps for Efficient 3D Human Generation [96.25056237689988]
  3D generative adversarial networks (GANs) have demonstrated state-of-the-art (SOTA) quality and diversity for generated assets. Current 3D GAN architectures, however, rely on volume representations, which are slow to render, thereby hampering the GAN training and requiring multi-view-inconsistent 2D upsamplers.
  arXiv  Detail & Related papers  (2023-11-29T18:04:07Z)
• Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data [56.49494318285391]
  We present Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving. This is done by learning to translate randomized simulation images into simulated segmentation and depth maps. This allows us to train an end-to-end RL policy in simulation, and directly deploy in the real-world.
  arXiv  Detail & Related papers  (2022-10-25T17:50:36Z)
• Recovering and Simulating Pedestrians in the Wild [81.38135735146015]
  We propose to recover the shape and motion of pedestrians from sensor readings captured in the wild by a self-driving car driving around. We incorporate the reconstructed pedestrian assets bank in a realistic 3D simulation system. We show that the simulated LiDAR data can be used to significantly reduce the amount of real-world data required for visual perception tasks.
  arXiv  Detail & Related papers  (2020-11-16T17:16:32Z)
• Photorealism in Driving Simulations: Blending Generative Adversarial Image Synthesis with Rendering [0.0]
  We introduce a hybrid generative neural graphics pipeline for improving the visual fidelity of driving simulations. We form 2D semantic images from 3D scenery consisting of simple object models without textures. These semantic images are then converted into photorealistic RGB images with a state-of-the-art Generative Adrial Network (GAN) trained on real-world driving scenes.
  arXiv  Detail & Related papers  (2020-07-31T03:25:17Z)
• LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World [84.57894492587053]
  We develop a novel simulator that captures both the power of physics-based and learning-based simulation. We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation. We showcase LiDARsim's usefulness for perception algorithms-testing on long-tail events and end-to-end closed-loop evaluation on safety-critical scenarios.
  arXiv  Detail & Related papers  (2020-06-16T17:44:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.