WM-Cov: Test Adequacy for Interactive World-Model-Style Autonomous Driving Simulation

• URL: http://arxiv.org/abs/2608.00298v1
• Date: Fri, 31 Jul 2026 21:16:20 GMT
• Title: WM-Cov: Test Adequacy for Interactive World-Model-Style Autonomous Driving Simulation
• Abstract summary: World models and generative simulators are emerging as interactive testing infrastructure for autonomous driving.<n>This paper formulates interactive world-model-style testing adequacy and introduces WM-Cov.<n> WM-Cov reports adequacy through coverage growth, valid-failure discovery, failure-mode diversity, realism, artifact suppression, and valid-evidence precision.
• Score: 0.596133719688107
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models and generative simulators are emerging as interactive testing infrastructure for autonomous driving because they can react to the ego planner and produce counterfactual, rare, and safety-critical rollouts. This changes a test scenario from a fixed replayed trajectory into an interactive scenario family whose realized evolution depends on the planner under test. The unresolved question is therefore not only whether dangerous rollouts can be generated, but what valid closed-loop evidence is enough to support a specified testing intent and stopping decision. This paper formulates interactive world-model-style testing adequacy and introduces WM-Cov, a provider-agnostic evaluation layer that converts raw provider outputs into requested, realized, and valid evidence. WM-Cov reports adequacy through coverage growth, valid-failure discovery, failure-mode diversity, realism, artifact suppression, duplicate accounting, and valid-evidence precision. Studies on executed TeraSim/SUMO events, WM-like mixed trace pools, and a real DriveArena TrafficManager--WorldDreamer matrix show that dangerous-looking events can include valid ADS failures, duplicates, partial realizations, and artifacts. The DriveArena matrix evaluates two planners, two horizons, six prompt conditions, and 360 ego-route requests; 304 attempts become fully realized evidence and 56 remain partial. A disjoint 80-request route-slice check yields 74 fully realized and 6 partial attempts. The results support evaluating world-model-style testing by convergence of valid interactive evidence under budget, rather than by raw generated failures or prompt coverage alone.

Related papers

• HarnessEval-W: Agentifying the Evaluation of Visual Worlds [149.63310396189317]
  HarnessEval-W is an agentified evaluation pipeline for benchmarking world models.<n>Rather than applying a fixed rubric, HarnessEval-W decomposes the evaluation question into measurable subproblems.<n>We apply HarnessEval-W to 18 representative world models over 330 evaluation cases.
  arXiv  Detail & Related papers  (2026-08-17T17:43:24Z)
• How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks [6.001213285107465]
  AutoResearchEval features 100 tasks grounded in published frontier science across 7 scientific domains.<n> evaluating 8 harness-model combinations yields 800 autoresearch agent trajectories.<n>We leverage a human-calibrated agent-as-a-judge pipeline to inspect complete trajectories and intermediate artifacts.
  arXiv  Detail & Related papers  (2026-08-14T21:39:38Z)
• Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators [2.299872239734834]
  We argue that any World Model (WM) used as a test oracle must first be accredited before its verdicts can serve as evidence.<n>Our framework is embodiment-agnostic, and is instantiated in autonomous driving.
  arXiv  Detail & Related papers  (2026-07-08T09:30:31Z)
• ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Driving [56.96697737229895]
  Reactive capability is a key property of data-driven behavior world model simulators for autonomous driving simulation systems.<n>We introduce ReactSim-Bench for evaluating the reactive capability of behavior world model simulation in autonomous driving.
  arXiv  Detail & Related papers  (2026-06-12T03:11:06Z)
• What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs [52.50210189669399]
  Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs)<n>We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation.<n>We show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods.
  arXiv  Detail & Related papers  (2026-06-01T03:18:01Z)
• Can Attribution Predict Risk? From Multi-View Attribution to Planning Risk Signals in End-to-End Autonomous Driving [51.726472686971896]
  End-to-end autonomous driving models generate future trajectories from multi-view inputs.<n> Existing methods fail to reveal the visual evidence underlying trajectory generation.<n>We propose a hierarchical attribution framework for end-to-end planning.
  arXiv  Detail & Related papers  (2026-05-07T13:38:18Z)
• Dynasto: Validity-Aware Dynamic-Static Parameter Optimization for Autonomous Driving Testing [6.969678532857503]
  We propose Dynasto, a two-step testing approach to uncover safety-critical failures in autonomous driving systems.<n>First, we train an adversarial agent using reinforcement learning with temporal-logic-based validity criteria.<n>Second, a genetic algorithm searches over initial conditions while replaying the adversary's failure-inducing behaviors to reveal additional failures that the RL agent alone does not uncover.
  arXiv  Detail & Related papers  (2026-03-22T22:35:30Z)
• CRASH: Cognitive Reasoning Agent for Safety Hazards in Autonomous Driving [0.3823356975862005]
  This work examines real-world AV incidents reported in the NHTSA database.<n>We introduce CRASH, an agent that automates reasoning over crash reports by leveraging both standardized fields and unstructured narrative descriptions.<n>Our findings show that CRASH attributes 64% of incidents to perception or planning failures, underscoring the importance of reasoning-based analysis for accurate fault attribution.<n>We further validate CRASH with five domain experts, achieving 86% accuracy in attributing AV system failures.
  arXiv  Detail & Related papers  (2026-03-16T14:39:33Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• From Accidents to Insights: Leveraging Multimodal Data for Scenario-Driven ADS Testing [3.984220091774453]
  This paper introduces TRACE, a scenario-based ADS Test case Generation framework for Critical Scenarios.<n>By leveraging multimodal data to extract challenging scenarios from real-world car crash reports, TRACE constructs numerous critical test cases with less data.<n>User feedback reveals that TRACE demonstrates superior scenario reconstruction accuracy, with 77.5% of the scenarios being rated as'mostly or 'totally' consistent.
  arXiv  Detail & Related papers  (2025-02-04T05:21:29Z)
• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [65.24988062003096]
  We present NAVSIM, a framework for benchmarking vision-based driving policies. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights.
  arXiv  Detail & Related papers  (2024-06-21T17:59:02Z)
• Zero-Knowledge Proof of Traffic: A Deterministic and Privacy-Preserving Cross Verification Mechanism for Cooperative Perception Data [7.919995199824006]
  This study proposes a novel approach called zero-knowledge Proof of Traffic (zk-PoT) Multiple independent proofs regarding the same vehicle can be deterministically cross-verified by any receivers without relying on ground truth, probabilistic, or plausibility evaluations. The proposed approach can be incorporated into existing works, including the European Telecommunications Standards Institute (ETSI) and the International Organization for Standardization (ISO) ITS standards.
  arXiv  Detail & Related papers  (2023-12-13T07:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.