How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models

• URL: http://arxiv.org/abs/2608.28404v1
• Date: Fri, 28 Aug 2026 14:56:01 GMT
• Title: How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models
• Abstract summary: Video generation for autonomous driving is expensive to collect, bound by privacy requirements, and cannot be scraped at will.<n>We present a systematic scaling-law study of video diffusion models trained from scratch on driving data.<n>We train the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation.
• Score: 57.34034448977433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video generation for autonomous driving cannot follow the web-scale route: driving data is expensive to collect, bound by privacy requirements, and cannot be scraped at will, so models must make the most of a fixed corpus. We present a systematic scaling-law study of video diffusion models trained from scratch on driving data: a family of models from 1M to 9B parameters, trained at different exposures on up to 5,500 hours of driving. Validation loss follows consistent power laws in both model size and training exposure, answering the questions that shape a training budget: whether compute is better spent on longer training or on a larger model, and whether more data is needed. Loss improves much faster with training exposure than with model size, making longer training the most effective way to improve a fixed model under limited compute. However, larger models continue to achieve lower asymptotic loss, so compute-optimal scaling still favors increasing model size when sufficient compute and data are available. Guided by these laws, we train a 9B-parameter model, to our knowledge the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation, as measured on nuScenes. Our code and pretrained models are available at https://github.com/valeoai/VATIX. NATIX is separately releasing the underlying driving data in stages.

Related papers

• Abra: Scaling Diffusion Image Training [19.77608057281383]
  We present a systematic scaling law study for text-to-image diffusion models.<n>We show that diffusion models scale just as predictably as language models but require far more data to train optimally.<n>We show that unlike language models, diffusion models are robust to overtraining and that practitioners should err on the side of more data rather than a larger model.
  arXiv  Detail & Related papers  (2026-08-18T02:36:25Z)
• Compute-Optimal Scaling for Value-Based Deep RL [99.680827753493]
  We investigate compute scaling for online, value-based deep RL.<n>Our analysis reveals a nuanced interplay between model size, batch size, and UTD.<n>We provide a mental model for understanding this phenomenon and build guidelines for choosing batch size and UTD.
  arXiv  Detail & Related papers  (2025-08-20T17:54:21Z)
• Training and Inference Efficiency of Encoder-Decoder Speech Models [25.031622057759492]
  We focus on the efficiency angle and ask the questions of whether we are training these speech models efficiently.<n>We show that negligence in mini-batch sampling leads to more than 50% being spent on padding.<n>We find that adjusting the model architecture to transfer model parameters from the decoder to the encoder results in a 3x inference speedup.
  arXiv  Detail & Related papers  (2025-03-07T20:57:43Z)
• Scaling Inference-Efficient Language Models [3.271571137474847]
  We show that model architecture affects inference latency, where models of the same size can have up to 3.5x difference in latency.<n>We modify the Chinchilla scaling laws to co-optimize the model parameter count, the number of training tokens, and the model architecture.<n>We release the Morph-1B model, which improves inference latency by 1.8x while maintaining accuracy on downstream tasks.
  arXiv  Detail & Related papers  (2025-01-30T03:16:44Z)
• A Hitchhiker's Guide to Scaling Law Estimation [56.06982415792523]
  Scaling laws predict the loss of a target machine learning model by extrapolating from easier-to-train models with fewer parameters or smaller training sets.<n>We estimate more than 1000 scaling laws, then derive a set of best practices for estimating scaling laws in new model families.
  arXiv  Detail & Related papers  (2024-10-15T17:59:10Z)
• A Dynamical Model of Neural Scaling Laws [79.59705237659547]
  We analyze a random feature model trained with gradient descent as a solvable model of network training and generalization. Our theory shows how the gap between training and test loss can gradually build up over time due to repeated reuse of data.
  arXiv  Detail & Related papers  (2024-02-02T01:41:38Z)
• AI Model Disgorgement: Methods and Choices [127.54319351058167]
  We introduce a taxonomy of possible disgorgement methods that are applicable to modern machine learning systems. We investigate the meaning of "removing the effects" of data in the trained model in a way that does not require retraining from scratch.
  arXiv  Detail & Related papers  (2023-04-07T08:50:18Z)
• On Data Scaling in Masked Image Modeling [36.00347416479826]
  Masked image modeling (MIM) is suspected to be unable to benefit from larger data. Data scales ranging from 10% of ImageNet-1K to full ImageNet-22K, model sizes ranging from 49 million to 1 billion, and training lengths ranging from 125K iterations to 500K iterations. validation loss in pre-training is a good indicator to measure how well the model performs for fine-tuning on multiple tasks.
  arXiv  Detail & Related papers  (2022-06-09T17:58:24Z)
• Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers [94.43313684188819]
  We study the impact of model size in this setting, focusing on Transformer models for NLP tasks that are limited by compute. We first show that even though smaller Transformer models execute faster per iteration, wider and deeper models converge in significantly fewer steps. This leads to an apparent trade-off between the training efficiency of large Transformer models and the inference efficiency of small Transformer models.
  arXiv  Detail & Related papers  (2020-02-26T21:17:13Z)
• Scaling Laws for Neural Language Models [14.472857826717613]
  We study scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training.
  arXiv  Detail & Related papers  (2020-01-23T03:59:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.