ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios

• URL: http://arxiv.org/abs/2602.16073v1
• Date: Tue, 17 Feb 2026 22:57:43 GMT
• Title: ScenicRules: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios
• Authors: Kevin Kai-Chun Chang, Ekin Beyazit, Alberto Sangiovanni-Vincentelli, Tichakorn Wongpiromsarn, Sanjit A. Seshia,
• Abstract summary: We introduce ScenicRules, a benchmark for evaluating autonomous driving systems.<n>We first formalize a diverse set of objectives to serve as quantitative evaluation metrics.<n>We then construct a compact yet representative collection of scenarios spanning diverse driving contexts and near-accident situations.
• Score: 4.139042737793025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing autonomous driving systems for complex traffic environments requires balancing multiple objectives, such as avoiding collisions, obeying traffic rules, and making efficient progress. In many situations, these objectives cannot be satisfied simultaneously, and explicit priority relations naturally arise. Also, driving rules require context, so it is important to formally model the environment scenarios within which such rules apply. Existing benchmarks for evaluating autonomous vehicles lack such combinations of multi-objective prioritized rules and formal environment models. In this work, we introduce ScenicRules, a benchmark for evaluating autonomous driving systems in stochastic environments under prioritized multi-objective specifications. We first formalize a diverse set of objectives to serve as quantitative evaluation metrics. Next, we design a Hierarchical Rulebook framework that encodes multiple objectives and their priority relations in an interpretable and adaptable manner. We then construct a compact yet representative collection of scenarios spanning diverse driving contexts and near-accident situations, formally modeled in the Scenic language. Experimental results show that our formalized objectives and Hierarchical Rulebooks align well with human driving judgments and that our benchmark effectively exposes agent failures with respect to the prioritized objectives. Our benchmark can be accessed at https://github.com/BerkeleyLearnVerify/ScenicRules/.

Related papers

• DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving [34.19974984127512]
  We present DriveCombo, a text-based benchmark for compositional traffic rule reasoning.<n>Inspired by human drivers' cognitive development, we propose a systematic Five-Level Cognitive Ladder.<n>We also propose a Rule2Scene Agent that maps language-based traffic rules to dynamic driving scenes.
  arXiv  Detail & Related papers  (2026-03-02T09:12:40Z)
• DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models [24.168614747778538]
  We introduce DriveCritic, a novel framework featuring two key contributions.<n>The dataset is a curated collection of challenging scenarios where context is critical for correct judgment.<n>The DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context.
  arXiv  Detail & Related papers  (2025-10-15T03:00:38Z)
• Steerable Adversarial Scenario Generation through Test-Time Preference Alignment [58.37104890690234]
  Adversarial scenario generation is a cost-effective approach for safety assessment of autonomous driving systems.<n>We introduce a new framework named textbfSteerable textbfAdversarial scenario textbfGEnerator (SAGE)<n>SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining.
  arXiv  Detail & Related papers  (2025-09-24T13:27:35Z)
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving [64.12414815634847]
  Vision-Language Models (VLMs) and Driving World Models (DWMs) have independently emerged as powerful recipes addressing different aspects of this challenge.<n>We propose ImagiDrive, a novel end-to-end autonomous driving framework that integrates a VLM-based driving agent with a DWM-based scene imaginer.
  arXiv  Detail & Related papers  (2025-08-15T12:06:55Z)
• DRIVE: Dynamic Rule Inference and Verified Evaluation for Constraint-Aware Autonomous Driving [37.24058519921229]
  We present DRIVE, a novel framework for Dynamic Rule Inference and Verified Evaluation.<n>D DRIVE achieves 0.0% soft constraint violation rates, smoother trajectories, and stronger generalization across diverse driving scenarios.<n> Verified evaluations further demonstrate the efficiency, explanability, and robustness of the framework for real-world deployment.
  arXiv  Detail & Related papers  (2025-08-06T03:56:06Z)
• A Framework for Learning Scoring Rules in Autonomous Driving Planning Systems [2.4578723416255754]
  FLoRA is a framework that learns interpretable scoring rules represented in temporal logic.<n>Our approach effectively learns to evaluate driving behavior even though the training data only contains positive examples.<n> Evaluations in closed-loop planning simulations demonstrate that our learned scoring rules outperform existing techniques.
  arXiv  Detail & Related papers  (2025-02-17T02:06:57Z)
• CRITERIA: a New Benchmarking Paradigm for Evaluating Trajectory Prediction Models for Autonomous Driving [6.868387710209245]
  We propose a new benChmarking paRadIgm for evaluaTing trajEctoRy predIction Approaches (CRITERIA) We show that the proposed benchmark can produce a more accurate ranking of the models and serve as a means of characterizing their behavior.
  arXiv  Detail & Related papers  (2023-10-11T18:28:15Z)
• Planning-oriented Autonomous Driving [60.93767791255728]
  We argue that a favorable framework should be devised and optimized in pursuit of the ultimate goal, i.e., planning of the self-driving car. We introduce Unified Autonomous Driving (UniAD), a comprehensive framework that incorporates full-stack driving tasks in one network.
  arXiv  Detail & Related papers  (2022-12-20T10:47:53Z)
• Decentralized Vehicle Coordination: The Berkeley DeepDrive Drone Dataset and Consensus-Based Models [76.32775745488073]
  We present a novel dataset and modeling framework designed to study motion planning in understructured environments.<n>We demonstrate that a consensus-based modeling approach can effectively explain the emergence of priority orders observed in our dataset.
  arXiv  Detail & Related papers  (2022-09-19T05:06:57Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding [46.52703817997932]
  Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians. We propose a model that synthesizes multiple input signals from the multimodal world. We show a significant performance improvement over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-06T13:59:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.