WayveScenes101: A Dataset and Benchmark for Novel View Synthesis in Autonomous Driving

• URL: http://arxiv.org/abs/2407.08280v1
• Date: Thu, 11 Jul 2024 08:29:45 GMT
• Title: WayveScenes101: A Dataset and Benchmark for Novel View Synthesis in Autonomous Driving
• Authors: Jannik Zürn, Paul Gladkov, Sofía Dudas, Fergal Cotter, Sofi Toteva, Jamie Shotton, Vasiliki Simaiaki, Nikhil Mohan,
• Abstract summary: WayveScenes101 is a dataset designed to help the community advance the state of the art in novel view synthesis. The dataset comprises 101 driving scenes across a wide range of environmental conditions and driving scenarios.
• Score: 4.911903454560829
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present WayveScenes101, a dataset designed to help the community advance the state of the art in novel view synthesis that focuses on challenging driving scenes containing many dynamic and deformable elements with changing geometry and texture. The dataset comprises 101 driving scenes across a wide range of environmental conditions and driving scenarios. The dataset is designed for benchmarking reconstructions on in-the-wild driving scenes, with many inherent challenges for scene reconstruction methods including image glare, rapid exposure changes, and highly dynamic scenes with significant occlusion. Along with the raw images, we include COLMAP-derived camera poses in standard data formats. We propose an evaluation protocol for evaluating models on held-out camera views that are off-axis from the training views, specifically testing the generalisation capabilities of methods. Finally, we provide detailed metadata for all scenes, including weather, time of day, and traffic conditions, to allow for a detailed model performance breakdown across scene characteristics. Dataset and code are available at https://github.com/wayveai/wayve_scenes.

Related papers

• XLD: A Cross-Lane Dataset for Benchmarking Novel Driving View Synthesis [84.23233209017192]
  This paper presents a novel driving view synthesis dataset and benchmark specifically designed for autonomous driving simulations. The dataset is unique as it includes testing images captured by deviating from the training trajectory by 1-4 meters. We establish the first realistic benchmark for evaluating existing NVS approaches under front-only and multi-camera settings.
  arXiv  Detail & Related papers  (2024-06-26T14:00:21Z)
• SEVD: Synthetic Event-based Vision Dataset for Ego and Fixed Traffic Perception [22.114089372056238]
  We present SEVD, a first-of-its-kind multi-view ego, and fixed perception synthetic event-based dataset. SEVD spans urban, suburban, rural, and highway scenes featuring various classes of objects. We evaluate the dataset using state-of-the-art event-based (RED, RVT) and frame-based (YOLOv8) methods for traffic participant detection tasks.
  arXiv  Detail & Related papers  (2024-04-12T20:40:12Z)
• RSUD20K: A Dataset for Road Scene Understanding In Autonomous Driving [6.372000468173298]
  RSUD20K is a new dataset for road scene understanding, comprised of over 20K high-resolution images from the driving perspective on Bangladesh roads. Our work significantly improves upon previous efforts, providing detailed annotations and increased object complexity.
  arXiv  Detail & Related papers  (2024-01-14T16:10:42Z)
• Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting [32.59889755381453]
  Recent methods extend NeRF by incorporating tracked vehicle poses to animate vehicles, enabling photo-realistic view of dynamic urban street scenes. We introduce Street Gaussians, a new explicit scene representation that tackles these limitations. The proposed method consistently outperforms state-of-the-art methods across all datasets.
  arXiv  Detail & Related papers  (2024-01-02T18:59:55Z)
• On the Generation of a Synthetic Event-Based Vision Dataset for Navigation and Landing [69.34740063574921]
  This paper presents a methodology for generating event-based vision datasets from optimal landing trajectories. We construct sequences of photorealistic images of the lunar surface with the Planet and Asteroid Natural Scene Generation Utility. We demonstrate that the pipeline can generate realistic event-based representations of surface features by constructing a dataset of 500 trajectories.
  arXiv  Detail & Related papers  (2023-08-01T09:14:20Z)
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion [83.30168660888913]
  We present CommonScenes, a fully generative model that converts scene graphs into corresponding controllable 3D scenes. Our pipeline consists of two branches, one predicting the overall scene layout via a variational auto-encoder and the other generating compatible shapes. The generated scenes can be manipulated by editing the input scene graph and sampling the noise in the diffusion model.
  arXiv  Detail & Related papers  (2023-05-25T17:39:13Z)
• Traffic Scene Parsing through the TSP6K Dataset [109.69836680564616]
  We introduce a specialized traffic monitoring dataset, termed TSP6K, with high-quality pixel-level and instance-level annotations. The dataset captures more crowded traffic scenes with several times more traffic participants than the existing driving scenes. We propose a detail refining decoder for scene parsing, which recovers the details of different semantic regions in traffic scenes.
  arXiv  Detail & Related papers  (2023-03-06T02:05:14Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization [44.97567243883994]
  We propose a new benchmark for visual localization in outdoor scenes using crowd-sourced data. We show that our dataset is very challenging, with all evaluated methods failing on its hardest parts. As part of the dataset release, we provide the tooling used to generate it, enabling efficient and effective 2D correspondence annotation.
  arXiv  Detail & Related papers  (2021-09-09T19:25:48Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.