Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation

• URL: http://arxiv.org/abs/2607.06516v1
• Date: Tue, 07 Jul 2026 17:20:51 GMT
• Title: Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation
• Abstract summary: We present textbfemphPoint as Skeleton, a generative sensor simulation framework for state-updated autoregressive driving video generation.<n>To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation.<n>Experiments on nuScenes and nuPlan show that textitPoint as Skeleton improves autoregressive generation quality during closed-loop rollout.
• Score: 58.86413499077093
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes). We present \textbf{\emph{Point as Skeleton}}, a generative sensor simulation framework for state-updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that \textit{Point as Skeleton} improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at https://github.com/krauwu/point-as-skeleton.

Related papers

• CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis [3.0621944624163695]
  Corner-case generation is inherently a multi-case problem spanning visual representation, scene reasoning, and vehicle trajectory generation and control.<n>To unify these aspects within a single framework, we propose CARLA-GS, a modular corner-case synthesis pipeline.<n>This design enables high-level semantic reasoning, low-level physically executable motion, and corner-case generation within a unified pipeline.
  arXiv  Detail & Related papers  (2026-07-08T16:20:19Z)
• Cam2Sim: Neural Scenario Reconstruction for Closed-Loop Autonomous Driving Simulation [0.9114245816258512]
  Cam2Sim is a tool that transforms real-world driving recordings into playable CARLA simulation scenarios.<n>The framework supports ROS-based data extraction, parked-vehicle detection, OpenStreetMap-based map generation.<n>We validate Cam2Sim on a real-world urban-driving scenario with a camera-based end-to-end driving model.
  arXiv  Detail & Related papers  (2026-07-06T08:03:31Z)
• CausalDrive: Real-time Causal World Models for Autonomous Driving [60.66609721457312]
  We present CausalDrive, a controllable, real-time foundation driving world.<n>CaulDrive operates solely on the initial front-view frame, the ego-vehicle's trajectory, and a macroscopic text prompt.<n>We demonstrate its versatility across three downstream applications: (1) generative closed-loop evaluation with significantly mitigated collision artifacts, (2) large-scale Reinforcement Learning (RL) post-training driven by a Video2Reward module, and (3) real-time human-in-the-loop simulation.
  arXiv  Detail & Related papers  (2026-06-13T15:04:44Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation [26.98977914185036]
  An ideal traffic simulator replicates the realistic long-term point-to-point trip that a self-driving system experiences during deployment.<n>We propose InfGen, a unified next-token prediction model that performs interleaved closed-loop motion simulation and scene generation.<n>InfGen performs at the state-of-the-art in short-term (9s) traffic simulation, and significantly outperforms all other methods in long-term (30s) simulation.
  arXiv  Detail & Related papers  (2025-06-20T17:59:21Z)
• Pseudo-Simulation for Autonomous Driving [66.1981253104508]
  Existing evaluation paradigms for Autonomous Vehicles (AVs) face critical limitations.<n>Real-world evaluation is often challenging due to safety concerns and a lack of realism.<n>Open-loop evaluation relies on metrics that generally overlook compounding errors.
  arXiv  Detail & Related papers  (2025-06-04T17:57:53Z)
• Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model [63.336123527432136]
  We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation.<n>Unlike existing video generative models for autonomous driving, the proposed designs are tailored for interactive simulation.<n>We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-11T06:35:18Z)
• HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving [48.84595398410479]
  HUGSIM is a closed-loop, photo-realistic, and real-time simulator for evaluating autonomous driving algorithms.<n>We tackle challenges of novel view synthesis in closed-loop scenarios, including viewpoint extrapolation and 360-degree vehicle rendering.<n> HUGSIM offers a comprehensive benchmark across more than 70 sequences from KITTI-360, nuScenes, and PandaSet, along with over 400 varying scenarios.
  arXiv  Detail & Related papers  (2024-12-02T17:07:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.