Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

• URL: http://arxiv.org/abs/2607.14203v1
• Date: Wed, 15 Jul 2026 17:50:23 GMT
• Title: Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation
• Abstract summary: We present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a 3D world in a single forward pass.<n>The model accepts multi-view input from a camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections.<n>It reconstructs a 10-20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR that is 2.01 dB above the strongest evaluated baseline.
• Score: 63.04937864192328
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent years, neural simulation has become predominant, with methods such as NuRec playing a central role; however, these methods remain relatively slow and typically require per-scene tuning. In this work, we present Instant NuRec, a feed-forward neural reconstruction model that turns a short multi-view driving log into a fully simulatable 3D Gaussian Splatting (3DGS) world in a single forward pass. The model accepts multi-view input from a calibrated camera rig and emits a layered output consisting of static and dynamic 3DGS layers, a sky cubemap, and per-camera ISP corrections, while providing native support for non-pinhole camera models via 3DGUT. It reconstructs a 10-20-second multi-camera scene in roughly 1.5 seconds and achieves a PSNR on the Waymo Open Dataset that is 2.01 dB above the strongest evaluated baseline. Instant NuRec is deeply integrated into NuRec and is compatible with AlpaSim for closed-loop simulation.

Related papers

• Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation [58.86413499077093]
  We present textbfemphPoint as Skeleton, a generative sensor simulation framework for state-updated autoregressive driving video generation.<n>To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation.<n>Experiments on nuScenes and nuPlan show that textitPoint as Skeleton improves autoregressive generation quality during closed-loop rollout.
  arXiv  Detail & Related papers  (2026-07-07T17:20:51Z)
• Fast 3D Foundation Model Initialized Gaussian Splatting [0.04117494580521491]
  The proposed method produces ready-to-use 3DGS models at a fraction of the time required by existing pipelines.<n>The method is suitable for near real-time applications in robotics, VR, and autonomous navigation.
  arXiv  Detail & Related papers  (2026-07-03T11:24:27Z)
• MM-TRELLIS: Point-Cloud Guided Multi-Modal 3D Vehicle Generation in Autonomous Driving [48.042009630881275]
  MM-TRELLIS is a multi-modal version of TRELLIS for in-the-wild 3D vehicle generation.<n>It integrates LiDAR and image sensors from autonomous driving datasets into native 3D generative models.<n>Our method outperforms existing methods in high-fidelity 3D vehicle generation.
  arXiv  Detail & Related papers  (2026-06-23T08:31:39Z)
• Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation [63.01022057888141]
  Asset Harvester is an image-to-3D model and end-to-end pipeline that converts sparse, in-thewild object observations from real driving logs into complete, simulation-ready assets.<n>SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges.
  arXiv  Detail & Related papers  (2026-04-20T16:20:57Z)
• FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos [134.8812117202239]
  We present FunRec, a method for reconstructing functional 3D digital twins of indoor scenes directly from egocentric RGB-D interaction videos.<n>We demonstrate applications on URDF/USD export for simulation, hand-guided affordance mapping and robot-scene interaction.
  arXiv  Detail & Related papers  (2026-04-07T09:22:12Z)
• DGGT: Feedforward 4D Reconstruction of Dynamic Driving Scenes using Unposed Images [36.562825380568384]
  We introduce textbfDriving Gaussian Grounded Transformer (DGGT), a unified framework for pose-free dynamic scene reconstruction.<n>Our approach jointly predicts per-frame 3D Gaussian maps and camera parameters, disentangles dynamics with a lightweight dynamic head.<n>A diffusion-based rendering refinement further reduces motion/interpolation artifacts and improves novel-view quality under sparse inputs.
  arXiv  Detail & Related papers  (2025-12-02T18:29:18Z)
• SimULi: Real-Time LiDAR and Camera Simulation with Unscented Transforms [31.853574316696037]
  SimULi is the first method capable of rendering arbitrary camera models and LiDAR data in real-time.<n>Our method extends 3DGUT, which supports complex camera models, with LiDAR support, via an automated tiling strategy.<n>SimULi renders 10-20x faster than ray tracing approaches and 1.5-10x faster than priorization-based work.
  arXiv  Detail & Related papers  (2025-10-14T18:22:45Z)
• Hybrid Rendering for Multimodal Autonomous Driving: Merging Neural and Physics-Based Simulation [1.0027737736304287]
  We introduce a hybrid approach that combines the strengths of neural reconstruction with physics-based rendering.<n>Our approach significantly enhances novel view synthesis quality, especially for road surfaces and lane markings.<n>We achieve this by training a customized NeRF model on the original images with depth regularization derived from a noisy LiDAR point cloud.
  arXiv  Detail & Related papers  (2025-03-12T15:18:50Z)
• SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving [6.221538885604869]
  Existing neural radiance field (NeRF) methods for sensor-realistic rendering of camera and lidar data suffer from low rendering speeds.<n>We propose SplatAD, the first 3DGS-based method for realistic, real-time rendering of dynamic scenes for both camera and lidar data.
  arXiv  Detail & Related papers  (2024-11-25T16:18:22Z)
• DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features [65.8738034806085]
  DistillNeRF is a self-supervised learning framework for understanding 3D environments in autonomous driving scenes. Our method is a generalizable feedforward model that predicts a rich neural scene representation from sparse, single-frame multi-view camera inputs.
  arXiv  Detail & Related papers  (2024-06-17T21:15:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.