TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

• URL: http://arxiv.org/abs/2607.13028v1
• Date: Tue, 14 Jul 2026 17:59:02 GMT
• Title: TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale
• Abstract summary: TerraZero is a procedural driving simulator and self-play training stack.<n>A C engine runs simulation on the CPU and policy inference on the GPU over a zero-copy path.<n>Every reported policy trains from scratch by reinforcement learning alone on a compute-efficient self-play recipe.
• Score: 18.077282979391423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training robust autonomous driving agents requires a simulator that is fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains. We present TerraZero, a procedural driving simulator and self-play training stack. A configurable C engine runs simulation on the CPU and policy inference on the GPU over a zero-copy path, sustaining 1.3M agent-steps per second on a single server-grade GPU, far faster than existing object-level simulators, while keeping fidelity lighter single-agent systems omit: heterogeneous agents, multiple dynamics models, and full traffic-rule enforcement. TerraZero treats logged data only as a source of real-world map geometry, populating each map with randomized rule-based road users and signal controllers and randomizing agent dynamics, rewards, and sizes per episode, so a map yields an unbounded set of scenarios. Every reported policy trains from scratch by reinforcement learning alone on a compute-efficient self-play recipe across GPUs, with zero human demonstrations and no fallback planner at inference. Policies generalize zero-shot across cities and datasets, including emergent left-hand-traffic driving without explicit supervision. As an ego policy, TerraZero is the first fully learned policy to top the InterPlan long-tail benchmark, ahead of larger learned planners; on routine-driving val14 it ranks among the best approaches and is the safest, posting the best collision and time-to-collision scores. On Waymo Open Sim Agents realism the same recipe outperforms other demonstration-free methods and is competitive with the strongest reference-anchored self-play method. One stack serves both roles: driving policies across dynamics for cars and trucks, and sim agents that jointly control vehicles, pedestrians, and cyclists.

Related papers

• Learning Sim-to-Real Humanoid Locomotion in 15 Minutes [51.500643119683225]
  We introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3.<n>Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments.<n>We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots.
  arXiv  Detail & Related papers  (2025-12-01T18:55:17Z)
• SPACeR: Self-Play Anchoring with Centralized Reference Models [50.55045557371374]
  Sim agent policies are realistic, human-like, fast, and scalable in multi-agent settings.<n>Recent progress in imitation learning with large diffusion-based or tokenized models has shown that behaviors can be captured directly from human driving data.<n>We propose SPACeR, a framework that leverages a pretrained tokenized autoregressive motion model as a central reference policy.
  arXiv  Detail & Related papers  (2025-10-20T19:53:02Z)
• Learning to Drive from a World Model [0.5055815271772576]
  We propose an end-to-end training architecture that uses real driving data to train a driving policy in an on-policy simulator.<n>We show two different methods of simulation, one with reprojective simulation and one with a learned world model.<n>We evaluate the performance of these policies in a closed-loop simulation and when deployed in a real-world advanced driver-assistance system.
  arXiv  Detail & Related papers  (2025-04-27T02:17:22Z)
• Building reliable sim driving agents by scaling self-play [3.3378669626639423]
  Simulation agents are essential for designing and testing systems that interact with humans, such as autonomous vehicles (AVs)<n>We propose scaling self-play to thousands of scenarios on the Open Motion dataset under semi-realistic limits on human perception and control.<n>We generalize to unseen test scenes, achieving a 99.8% goal completion rate with less than 0.8% combined collision and off-road incidents.
  arXiv  Detail & Related papers  (2025-02-20T16:30:45Z)
• Robust Autonomy Emerges from Self-Play [63.44745854476206]
  We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale.<n>This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour.
  arXiv  Detail & Related papers  (2025-02-05T16:41:05Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [57.278726604424556]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers.<n>Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy.<n>We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• WROOM: An Autonomous Driving Approach for Off-Road Navigation [17.74237088460657]
  We design an end-to-end reinforcement learning (RL) system for an autonomous vehicle in off-road environments. We warm-start the agent by imitating a rule-based controller and utilize Proximal Policy Optimization (PPO) to improve the policy. We propose a novel simulation environment to replicate off-road driving scenarios and deploy our proposed approach on a real buggy RC car.
  arXiv  Detail & Related papers  (2024-04-12T23:55:59Z)
• Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data [56.49494318285391]
  We present Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving. This is done by learning to translate randomized simulation images into simulated segmentation and depth maps. This allows us to train an end-to-end RL policy in simulation, and directly deploy in the real-world.
  arXiv  Detail & Related papers  (2022-10-25T17:50:36Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.