PairedGTA: Generating Driving Datasets for Controlled Photometric Shift Analysis

• URL: http://arxiv.org/abs/2606.01192v1
• Date: Sun, 31 May 2026 12:13:19 GMT
• Title: PairedGTA: Generating Driving Datasets for Controlled Photometric Shift Analysis
• Abstract summary: This work introduces a data generation framework based on a high-fidelity game engine for extracting perfectly paired images.<n>For each sampled location, it procedurally instantiates dynamic entities and renders pixel-aligned images under diverse adverse conditions.<n>The benefit of the proposed generation framework in driving scenarios is demonstrated through a systematic analysis of semantic segmentation models.
• Score: 43.25348855116086
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Evaluating the performance of visual perception systems for autonomous driving is essential to ensure reliable operation across diverse environmental scenarios. Ideally, a balanced and fair analysis across different adverse conditions would require perfectly paired images of the same scene under different weather or illumination changes. This would allow evaluating the effect of photometric shifts independently of geometry and semantic changes. Unfortunately, real-world datasets rarely provide images of the same scene under different environmental conditions, because, normally, camera pose, traffic, and locations of dynamic objects (vehicles, pedestrians, etc.) vary over time, thus yielding only coarsely paired data. To address this challenge, this work introduces a data generation framework based on a high-fidelity game engine for extracting perfectly paired images. By leveraging software APIs that communicate with the GTA game engine, the framework modifies illumination and weather conditions while preserving scene geometry, camera pose, and the identity and placement of dynamic objects. For each sampled location, it procedurally instantiates dynamic entities and renders pixel-aligned images under diverse adverse conditions. The benefit of the proposed generation framework in driving scenarios is demonstrated through a systematic analysis of semantic segmentation models, whose output degradation can be attributed more directly to photometric shifts rather than to uncontrolled semantic or geometric factors.

Related papers

• What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning [2.5264159985739623]
  We propose a framework that systematically removes individual detected objects from front-camera images.<n>This isolates the causal effect of each object's presence on the model's planning behaviour.<n>We also find cases where the model responds strongly to objects a human driver would consider irrelevant.
  arXiv  Detail & Related papers  (2026-07-18T19:20:16Z)
• RealEngine: Simulating Autonomous Driving in Realistic Context [60.55873455475112]
  RealEngine is a novel driving simulation framework that holistically integrates 3D scene reconstruction and novel view synthesis techniques.<n>By leveraging real-world multi-modal sensor data, RealEngine reconstructs background scenes and foreground traffic participants separately, allowing for highly diverse and realistic traffic scenarios.<n>RealEngine supports three essential driving simulation categories: non-reactive simulation, safety testing, and multi-agent interaction.
  arXiv  Detail & Related papers  (2025-05-22T17:01:00Z)
• Drive-1-to-3: Enriching Diffusion Priors for Novel View Synthesis of Real Vehicles [81.29018359825872]
  This paper consolidates a set of good practices to finetune large pretrained models for a real-world task.<n>Specifically, we develop several strategies to account for discrepancies between the synthetic data and real driving data.<n>Our insights lead to effective finetuning that results in a $68.8%$ reduction in FID for novel view synthesis over prior arts.
  arXiv  Detail & Related papers  (2024-12-19T03:39:13Z)
• WayveScenes101: A Dataset and Benchmark for Novel View Synthesis in Autonomous Driving [4.911903454560829]
  WayveScenes101 is a dataset designed to help the community advance the state of the art in novel view synthesis. The dataset comprises 101 driving scenes across a wide range of environmental conditions and driving scenarios.
  arXiv  Detail & Related papers  (2024-07-11T08:29:45Z)
• ConstScene: Dataset and Model for Advancing Robust Semantic Segmentation in Construction Environments [1.4070907500169874]
  This paper introduces a new semantic segmentation dataset specifically tailored for construction sites. The dataset is designed to enhance the training and evaluation of object detection models.
  arXiv  Detail & Related papers  (2023-12-27T10:49:19Z)
• Multi-Object Manipulation via Object-Centric Neural Scattering Functions [40.45919680959231]
  We propose using object-centric neural scattering functions (OSFs) as object representations in a model-predictive control framework. OSFs model per-object light transport, enabling compositional scene re-rendering under object rearrangement and varying lighting conditions.
  arXiv  Detail & Related papers  (2023-06-14T21:14:10Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition [81.24107630746508]
  We present GeoSim, a geometry-aware image composition process that synthesizes novel urban driving scenes. We first build a diverse bank of 3D objects with both realistic geometry and appearance from sensor data. The resulting synthetic images are photorealistic, traffic-aware, and geometrically consistent, allowing image simulation to scale to complex use cases.
  arXiv  Detail & Related papers  (2021-01-16T23:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.