RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos

• URL: http://arxiv.org/abs/2606.16278v1
• Date: Mon, 15 Jun 2026 06:33:13 GMT
• Title: RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos
• Abstract summary: We propose a structure-preserving and asset-aware Sim-to-Real framework for edited 3DGS driving videos.<n>RealityBridge uses multimodal controls, including rendered videos, foreground masks, edge maps, and semantic masks, together with a lightweight GateNet.<n>Experiments show that RealityBridge outperforms existing methods in artifact removal, illumination, and long-sequence temporal consistency.
• Score: 40.2251078105003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-tail hazardous scenarios are essential for safety-oriented autonomous driving, yet they are difficult to collect and reproduce at scale. Editable 3D Gaussian Splatting (3DGS) simulation offers a promising alternative by reconstructing real driving scenes and supporting controllable scene editing. However, edited 3DGS-rendered videos still suffer from a significant Sim-to-Real gap, including rendering artifacts, degraded foreground assets, inconsistent illumination, and temporal flickering. Existing restoration and video generation methods are insufficient for this task, as they often fail to jointly repair 3DGS-specific artifacts, improve visual realism, and ensure temporal consistency. To fill this gap, we propose RealityBridge, a structure-preserving and asset-aware Sim-to-Real framework for edited 3DGS driving videos. RealityBridge uses multimodal controls, including rendered videos, foreground masks, edge maps, and semantic masks, together with a lightweight GateNet for adaptive condition allocation across backbone layers. We further construct targeted training data and introduce autoregressive long-video training with reward-guided post-training to improve restoration quality, temporal stability, and hallucination suppression. Extensive experiments on internal and public driving datasets show that RealityBridge outperforms existing methods in artifact removal, illumination harmonization, and long-sequence temporal consistency.

Related papers

• RealVDeblur: One-Step Diffusion for Generalizable Real-World Video Deblurring [38.962439175751975]
  Real-world video deblurring remains challenging due to diverse motion patterns, complex degradations, and the scarcity of realistic training data.<n>This work presents textbfRealVDeblur, an efficient generative framework designed to improve in-the-wild robustness under diverse real capture conditions.
  arXiv  Detail & Related papers  (2026-07-22T18:01:03Z)
• DriveCtrl: Conditioned Sim-to-Real Driving Video Generation [16.424889754682727]
  DriveCtrl is a controllable sim-to-real video generation framework for realistic driving video synthesis.<n>We introduce a scalable data generation pipeline that transforms simulator videos into realistic driving footage matching the visual style of a target real-world dataset.
  arXiv  Detail & Related papers  (2026-05-14T17:29:35Z)
• CityRAG: Stepping Into a City via Spatially-Grounded Video Generation [61.843558834675065]
  We present CityRAG, a video generative model that generates a 3D-consistent, navigable environment that is spatially grounded.<n>CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes.<n>Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
  arXiv  Detail & Related papers  (2026-04-21T17:59:03Z)
• Lyra 2.0: Explorable Generative 3D Worlds [77.45279013687427]
  We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale.<n>To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing.<n>We train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it.
  arXiv  Detail & Related papers  (2026-04-14T17:59:44Z)
• RealMaster: Lifting Rendered Scenes into Photorealistic Video [55.04231137698114]
  State-of-the-art video generation models produce remarkable photorealism, but they lack the precise control required to align generated content with scene requirements.<n>We present RealMaster, a method that leverages video diffusion models to lift rendered video into photorealistic video while maintaining full alignment with the output of the 3D engine.<n>RealMaster significantly outperforms existing video editing baselines, improving photorealism while preserving the geometry, dynamics, and identity specified by the original 3D control.
  arXiv  Detail & Related papers  (2026-03-24T17:32:42Z)
• HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles [63.88996084630768]
  Controllable driving scene generation is critical for realistic and scalable autonomous driving simulation.<n>We introduce HorizonForge, a unified framework that reconstructs scenes as editable Gaussian Splats and Meshes.<n>Experiments show that Gaussian-Mesh representation delivers substantially higher fidelity than alternative 3D representations.
  arXiv  Detail & Related papers  (2026-02-24T20:03:47Z)
• Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes [22.450051108066216]
  We present C2R (Coarse-to-Real), a generative framework that synthesizes real-style urban crowd videos.<n>Our approach uses coarse 3D renderings to explicitly control scene layout, camera motion, and human trajectories.<n>It produces temporally consistent, controllable, and realistic urban scene videos from minimal 3D input.
  arXiv  Detail & Related papers  (2026-01-29T20:29:04Z)
• Mirage: One-Step Video Diffusion for Photorealistic and Coherent Asset Editing in Driving Scenes [11.72942978943414]
  Vision-centric autonomous driving systems rely on diverse and scalable training data to achieve robust performance.<n>Video object editing offers a promising path for data augmentation.<n>Existing methods often struggle to maintain both high visual fidelity and temporal coherence.<n>Mirage builds upon a text-to-video diffusion prior to ensure temporal consistency across frames.
  arXiv  Detail & Related papers  (2025-12-30T13:40:23Z)
• SceneCrafter: Controllable Multi-View Driving Scene Editing [44.91248700043744]
  We propose SceneCrafter, a versatile editor for realistic 3D-consistent manipulation of driving scenes captured from multiple cameras.<n>SceneCrafter achieves state-of-the-art realism, controllability, 3D consistency, and scene editing quality compared to existing baselines.
  arXiv  Detail & Related papers  (2025-06-24T10:23:47Z)
• Enhancing Temporal Consistency in Video Editing by Reconstructing Videos with 3D Gaussian Splatting [94.84688557937123]
  Video-3DGS is a 3D Gaussian Splatting (3DGS)-based video refiner designed to enhance temporal consistency in zero-shot video editors.<n>Our approach utilizes a two-stage 3D Gaussian optimizing process tailored for editing dynamic monocular videos.<n>It enhances video editing by ensuring temporal consistency across 58 dynamic monocular videos.
  arXiv  Detail & Related papers  (2024-06-04T17:57:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.