Risk-Controllable Multi-View Diffusion for Driving Scenario Generation

• URL: http://arxiv.org/abs/2603.11534v1
• Date: Thu, 12 Mar 2026 04:44:12 GMT
• Title: Risk-Controllable Multi-View Diffusion for Driving Scenario Generation
• Abstract summary: Long-tail risky situations are rarely observed in real-world data and difficult to specify through manual scenario design.<n>RiskMV-DPO is a general and systematic pipeline for physically-informed, risk-controllable multi-view scenario generation.<n>Experiments show that RiskMV-DPO can freely generate a wide spectrum of diverse long-tail scenarios while maintaining state-of-the-art visual quality.
• Score: 25.426382365795444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating safety-critical driving scenarios is crucial for evaluating and improving autonomous driving systems, but long-tail risky situations are rarely observed in real-world data and difficult to specify through manual scenario design. Existing generative approaches typically treat risk as an after-the-fact label and struggle to maintain geometric consistency in multi-view driving scenes. We present RiskMV-DPO, a general and systematic pipeline for physically-informed, risk-controllable multi-view scenario generation. By integrating target risk levels with physically-grounded risk modeling, we autonomously synthesize diverse and high-stakes dynamic trajectories that serve as explicit geometric anchors for a diffusion-based video generator. To ensure spatial-temporal coherence and geometric fidelity, we introduce a geometry-appearance alignment module and a region-aware direct preference optimization (RA-DPO) strategy with motion-aware masking to focus learning on localized dynamic regions.Experiments on the nuScenes dataset show that RiskMV-DPO can freely generate a wide spectrum of diverse long-tail scenarios while maintaining state-of-the-art visual quality, improving 3D detection mAP from 18.17 to 30.50 and reducing FID to 15.70. Our work shifts the role of world models from passive environment prediction to proactive, risk-controllable synthesis, providing a scalable toolchain for the safety-oriented development of embodied intelligence.

Related papers

• Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Driving [82.69496624372944]
  "Only driving like the expert" suffers from limited generalization.<n>Can an E2E-AD system make reliable decisions without any expert action supervision?<n>We propose a unified framework named Risk-aware World Model Predictive Control.
  arXiv  Detail & Related papers  (2026-02-26T17:32:30Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Learning from Risk: LLM-Guided Generation of Safety-Critical Scenarios with Prior Knowledge [25.50999678115561]
  This paper presents a high-fidelity scenario generation framework that integrates a conditional variational autoencoder (CVAE) with a large language model (LLM)<n>Our framework substantially increases the coverage of high-risk and long-tail events, improves consistency between simulated and real-world traffic distributions, and exposes autonomous driving systems to interactions that are significantly more challenging than those produced by existing rule- or data-driven methods.
  arXiv  Detail & Related papers  (2025-11-25T09:53:09Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• RADE: Learning Risk-Adjustable Driving Environment via Multi-Agent Conditional Diffusion [17.46462636610847]
  Risk- Driving Environment (RADE) is a simulation framework that generates statistically realistic and risk-adjustable traffic scenes.<n>RADE learns risk-conditioned behaviors directly from data, preserving naturalistic multi-agent interactions with controllable risk levels.<n>We validate RADE on the real-world rounD dataset, demonstrating that it preserves statistical realism across varying risk levels.
  arXiv  Detail & Related papers  (2025-05-06T04:41:20Z)
• RiskNet: Interaction-Aware Risk Forecasting for Autonomous Driving in Long-Tail Scenarios [6.024186631622774]
  RiskNet is an interaction-aware risk forecasting framework for autonomous vehicles.<n>It integrates deterministic risk modeling with probabilistic behavior prediction for comprehensive risk assessment.<n>It supports real-time, scenario-adaptive risk forecasting and demonstrates strong generalization across uncertain driving environments.
  arXiv  Detail & Related papers  (2025-04-22T02:36:54Z)
• INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation [7.362380225654904]
  INSIGHT is a hierarchical vision-language model (VLM) framework designed to enhance hazard detection and edge-case evaluation.<n>By using multimodal data fusion, our approach integrates semantic and visual representations, enabling precise interpretation of driving scenarios.<n> Experimental results on the BDD100K dataset demonstrate a substantial improvement in hazard prediction straightforwardness and accuracy over existing models.
  arXiv  Detail & Related papers  (2025-02-01T01:43:53Z)
• ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable [88.08120417169971]
  Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner.
  arXiv  Detail & Related papers  (2024-09-12T08:26:33Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• I Know You Can't See Me: Dynamic Occlusion-Aware Safety Validation of Strategic Planners for Autonomous Vehicles Using Hypergames [12.244501203346566]
  We develop a novel multi-agent dynamic occlusion risk measure for assessing situational risk. We present a white-box, scenario-based, accelerated safety validation framework for assessing safety of strategic planners in AV.
  arXiv  Detail & Related papers  (2021-09-20T19:38:14Z)
• Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction [55.569050872780224]
  We present an online framework for safe crowd-robot interaction based on risk-sensitive optimal control, wherein the risk is modeled by the entropic risk measure. Our modular approach decouples the crowd-robot interaction into learning-based prediction and model-based control. A simulation study and a real-world experiment show that the proposed framework can accomplish safe and efficient navigation while avoiding collisions with more than 50 humans in the scene.
  arXiv  Detail & Related papers  (2020-09-12T02:02:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.