ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving

• URL: http://arxiv.org/abs/2605.21168v2
• Date: Mon, 25 May 2026 18:34:30 GMT
• Title: ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving
• Abstract summary: Safety-critical scenarios are central to evaluating autonomous driving systems.<n>Most scenario generation methods treat surrounding agents as adversaries.<n>We propose ScenePilot, a feasibility-guided framework that targets the boundary band.
• Score: 33.25871854514186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety-critical scenarios are central to evaluating autonomous driving systems, yet their rarity in naturalistic logs makes simulation-based stress testing indispensable. Most scenario generation methods treat surrounding agents as adversaries, but they either (i) induce failures without explicitly modeling vehicle-road physical limits, yielding visually extreme yet physically unsolvable crashes, or (ii) enforce physical feasibility or policy feasibility in isolation, which can over-focus on aggressive maneuvers or remain tied to a controller-dependent capability boundary. We propose ScenePilot, a feasibility-guided, boundary-driven framework that targets the boundary band: scenarios that are physically solvable in principle yet still cause the deployed autonomy stack to fail. We formulate generation as constrained multi-objective reinforcement learning, combining an RSS-derived physical-feasibility score $σ$ with an online-learned AV-risk predictor $Φ$, and introduce step-level feasibility-aware shielding to keep exploration near the feasibility boundary while avoiding infeasible artifacts. Experiments on SafeBench with multiple planners show that ScenePilot yields substantially higher collision rates (+6.2 percentage points) while preserving physical validity, and that adversarial fine-tuning on these boundary-band scenarios consistently reduces downstream crash rates. The code is available at https://github.com/QiyuRuan/ScenePilot.

Related papers

• End-to-end Conditional Diffusion for Realistic and Controllable Visual Traffic Scenario Generation [59.188006170485046]
  E2E-CDiff is an end-to-end conditional diffusion framework for controllable and realistic scenario generation.<n>Differentiable guidance regulates speed, enforces drivable-area compliance, and supports collision-avoidance or collision-seeking behaviors.<n>E2E-CDiff also performs competitively as a learning-based ego planner.
  arXiv  Detail & Related papers  (2026-07-21T02:13:25Z)
• CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation [12.01753684349527]
  We introduce Collision-Constrained Flow Matching (CCFM), a novel framework that guarantees precise collision control through hard physical constraints.<n>CCFM collision rate up to 46.4% on nuScenes and 83.1% on nuPlan, significantly outperforming baselines while preserving realistic driving behavior.
  arXiv  Detail & Related papers  (2026-07-05T18:26:14Z)
• World Engine: Towards the Era of Post-Training for Autonomous Driving [63.61277741443647]
  We introduce World Engine, a generative framework that reconstructs high-fidelity interactive environments from real-world logs.<n>World Engine substantially reduces failures in rare safety-critical scenarios and yields significantly larger gains than scaling pre-training data alone.<n>When deployed on a production-scale autonomous driving system, the resulting policy reduces simulated collisions.
  arXiv  Detail & Related papers  (2026-06-18T06:28:33Z)
• From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving [56.30087557121323]
  AlignADV is a learnability-guided closed-loop adversarial training framework.<n>It converts adversarial scenarios into resolvable and capability-aligned curricula.<n> Experiments show it reduces training steps by up to 40.6 percent.
  arXiv  Detail & Related papers  (2026-06-12T02:13:55Z)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving [62.43744546817599]
  Vision-language--action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings.<n>We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.<n> MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
  arXiv  Detail & Related papers  (2026-05-13T23:35:14Z)
• Self-Supervised Path Planning in Unstructured Environments via Global-Guided Differentiable Hard Constraint Projection [5.665217514922631]
  We propose a self-supervised framework for deploying deep learning agents for autonomous navigation in unstructured environments.<n>To mitigate data scarcity, we construct a Global-Guided Artificial Potential Field (G-APF), which provides dense supervision signals without manual labeling.<n> benchmarks on a test set of 20,000 scenarios demonstrate an 88.75% success rate, substantiating the enhanced operational safety.
  arXiv  Detail & Related papers  (2026-01-27T08:37:21Z)
• Controllable Collision Scenario Generation via Collision Pattern Prediction [22.736678908526653]
  We introduce controllable collision scenario generation, where the goal is to produce trajectories that realize a user-specified collision type and time-to-accident (TTA)<n>We propose a framework that predicts Collision Pattern, a compact and interpretable representation that captures the spatial configuration of the ego and the adversarial vehicles at impact.<n>Experiments show that our approach outperforms strong baselines in both collision rate and controllability.
  arXiv  Detail & Related papers  (2025-10-14T06:56:33Z)
• Steerable Adversarial Scenario Generation through Test-Time Preference Alignment [58.37104890690234]
  Adversarial scenario generation is a cost-effective approach for safety assessment of autonomous driving systems.<n>We introduce a new framework named textbfSteerable textbfAdversarial scenario textbfGEnerator (SAGE)<n>SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining.
  arXiv  Detail & Related papers  (2025-09-24T13:27:35Z)
• Foundation Models for Rapid Autonomy Validation [4.417336418010182]
  A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter.<n>We propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios.
  arXiv  Detail & Related papers  (2024-10-22T15:32:43Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.