Traffic Scenario Orchestration from Language via Constraint Satisfaction

• URL: http://arxiv.org/abs/2605.06966v1
• Date: Thu, 07 May 2026 21:34:27 GMT
• Title: Traffic Scenario Orchestration from Language via Constraint Satisfaction
• Abstract summary: We present a language-in, simulation-out scenario orchestrator for closed-loop testing AVs.<n>Under a benchmark of carefully crafted and diverse scenario descriptions, our approach greatly outperforms our baselines in orchestration success rate.
• Score: 23.997970363698027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous vehicles (AVs) require extensive testing in simulation, but test case generation for driving scenarios is laborious. The desired scenarios are often out-of-distribution and have precise requirements on interactions with the AV policy under test. Manually programming scenarios allows for precise controllability but is difficult to scale. On the other hand, statistical models can leverage compute and data, but struggle with precise controllability when out-of-distribution. We cast scenario orchestration as a constraint-solving problem and present a language-in, simulation-out scenario orchestrator for closed-loop testing AVs. Our approach leverages foundation model reasoning to translate general, natural language descriptions into a set of constraints as a scenario representation. This then allows us to leverage off the shelf solvers to solve for actor behaviors which meet precise testing intentions in closed-loop. Under a benchmark of carefully crafted and diverse scenario descriptions, our approach greatly outperforms our baselines in orchestration success rate. We further show that our closed-loop approach is especially important for scenarios which require ego-reactive specifications.

Related papers

• Scenario Generation for Testing of Autonomous Driving Systems Using Real-World Failure Records [19.330267460084773]
  We propose a scenario generation pipeline using categorical and contextual information available from historical records in natural language format.<n>We successfully apply our method to generate a diverse set of scenarios for testing autonomous navigation on Metadrive simulator.
  arXiv  Detail & Related papers  (2026-06-30T04:56:25Z)
• What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs [52.50210189669399]
  Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs)<n>We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation.<n>We show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods.
  arXiv  Detail & Related papers  (2026-06-01T03:18:01Z)
• Declarative Scenario-based Testing with RoadLogic [2.2107822884016404]
  We present RoadLogic that bridges declarative OS2 specifications and executable simulations.<n>Results show that RoadLogic consistently produces realistic, specification-satisfying simulations within minutes.
  arXiv  Detail & Related papers  (2026-03-10T10:11:09Z)
• An LLM-driven Scenario Generation Pipeline Using an Extended Scenic DSL for Autonomous Driving Safety Validation [4.602386383455713]
  Real-world crash reports are valuable for scenario-based testing of autonomous driving systems.<n>Current methods cannot effectively translate this multimodal data into precise, executable simulation scenarios.<n>We propose a scalable and verifiable pipeline that uses a large language model and a probabilistic intermediate representation.
  arXiv  Detail & Related papers  (2026-02-24T07:44:26Z)
• Steerable Adversarial Scenario Generation through Test-Time Preference Alignment [58.37104890690234]
  Adversarial scenario generation is a cost-effective approach for safety assessment of autonomous driving systems.<n>We introduce a new framework named textbfSteerable textbfAdversarial scenario textbfGEnerator (SAGE)<n>SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining.
  arXiv  Detail & Related papers  (2025-09-24T13:27:35Z)
• Exploring Semantic Clustering and Similarity Search for Heterogeneous Traffic Scenario Graph [41.2584175136191]
  We first propose an expressive and flexible heterogeneous,temporal graph model for representing traffic scenarios.<n>We then propose a self-supervised method to learn a universal embedding space for scenario graphs.<n>In particular, we implement contrastive learning alongside a bootstrapping-based approach and evaluate their suitability for the scenario space.
  arXiv  Detail & Related papers  (2025-07-07T15:10:03Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [102.1527101235251]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios.<n>By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors.<n>LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding [76.3876070043663]
  We propose DriveLMM-o1, a dataset and benchmark designed to advance step-wise visual reasoning for autonomous driving.<n>Our benchmark features over 18k VQA examples in the training set and more than 4k in the test set, covering diverse questions on perception, prediction, and planning.<n>Our model achieves a +7.49% gain in final answer accuracy, along with a 3.62% improvement in reasoning score over the previous best open-source model.
  arXiv  Detail & Related papers  (2025-03-13T17:59:01Z)
• Tree-Based Scenario Classification: A Formal Framework for Coverage Analysis on Test Drives of Autonomous Vehicles [0.0]
  In scenario-based testing, relevant (driving) scenarios are the basis of tests. We address the open challenges of classifying sets of scenarios and measuring coverage of theses scenarios in recorded test drives.
  arXiv  Detail & Related papers  (2023-07-11T08:30:57Z)
• Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior [135.78858513845233]
  STRIVE is a method to automatically generate challenging scenarios that cause a given planner to produce undesirable behavior, like collisions. To maintain scenario plausibility, the key idea is to leverage a learned model of traffic motion in the form of a graph-based conditional VAE. A subsequent optimization is used to find a "solution" to the scenario, ensuring it is useful to improve the given planner.
  arXiv  Detail & Related papers  (2021-12-09T18:03:27Z)
• Addressing the IEEE AV Test Challenge with Scenic and VerifAI [10.221093591444731]
  This paper summarizes our formal approach to testing autonomous vehicles (AVs) in simulation for the IEEE AV Test Challenge. We demonstrate a systematic testing framework leveraging our previous work on formally-driven simulation for intelligent cyber-physical systems.
  arXiv  Detail & Related papers  (2021-08-20T04:51:27Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Robust Question Answering Through Sub-part Alignment [53.94003466761305]
  We model question answering as an alignment problem. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.
  arXiv  Detail & Related papers  (2020-04-30T09:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.