Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators

• URL: http://arxiv.org/abs/2607.07196v1
• Date: Wed, 08 Jul 2026 09:30:31 GMT
• Title: Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
• Abstract summary: We argue that any World Model (WM) used as a test oracle must first be accredited before its verdicts can serve as evidence.<n>Our framework is embodiment-agnostic, and is instantiated in autonomous driving.
• Score: 2.299872239734834
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world, and returning a success or safety verdict. Yet a verdict is only as trustworthy as the WM that produced it, and the WM itself needs to be certified. In video-generation WMs, fidelity metrics such as Fréchet Video Distance (FVD) reward visual realism, but ignore whether the world responds correctly to the policy's actions, including those unseen in training. Classical simulation-based validation assumes a trusted simulator evaluating an untrusted policy, whereas generative WMs are themselves unverified learned artifacts. Hence, we argue that any WM used as a test oracle must first be accredited before its verdicts can serve as evidence. Building on credibility practices from safety-critical simulation, including Verification, Validation & Accreditation (VV&A), Safety of the Intended Functionality (SOTIF), and scenario-based testing standards, we define an admissibility ladder (L0-L4) that a WM must climb before its closed-loop verdicts are accepted as assurance evidence. Our framework is embodiment-agnostic, and is instantiated in autonomous driving (AD), where assurance methods for traditional simulation are most mature. Applied to two driving WMs, the lower rungs reveal a reversal: the model that ranks higher on visual generation quality (L0) ranks lower on action-following (L1-L2), so visual fidelity does not predict the action-robustness a closed-loop verdict depends on.

Related papers

• Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering [78.51705689800838]
  We propose an assurance-revision loop that uses deployment evidence to revise the requirements, model, or evaluator.<n>We then cast assured agentic development as a resource-allocation problem over human judgment, agent capability, and compute.
  arXiv  Detail & Related papers  (2026-09-10T17:58:48Z)
• SimVerity: When Does Simulated Agent Success Survive Physical Deployment? [5.081228499547384]
  We present SimVerity, a verdict-transfer assurance framework.<n>It replays matched scenarios on target smart home deployments.<n>It crosses-validates agent execution against independently qualified physical witnesses.
  arXiv  Detail & Related papers  (2026-08-25T19:00:49Z)
• WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation [85.15806404962193]
  Action-conditioned world models (ACWMs) promise to provide embodied AI with scalable predictive simulators for planning, policy evaluation, and data generation.<n>We evaluate ACWMs through the observable capabilities expected of physical simulators.<n>WorldSimProbe reveals systematic action-realization degradation across control variation, structured failures in interaction grounding and dynamics.
  arXiv  Detail & Related papers  (2026-08-10T08:48:06Z)
• WM-Cov: Test Adequacy for Interactive World-Model-Style Autonomous Driving Simulation [0.596133719688107]
  World models and generative simulators are emerging as interactive testing infrastructure for autonomous driving.<n>This paper formulates interactive world-model-style testing adequacy and introduces WM-Cov.<n> WM-Cov reports adequacy through coverage growth, valid-failure discovery, failure-mode diversity, realism, artifact suppression, and valid-evidence precision.
  arXiv  Detail & Related papers  (2026-07-31T21:16:20Z)
• A Causal Probabilistic Framework for Perception-Informed Closed-Loop Simulation of Autonomous Driving [1.9798595819903433]
  Software-in-the-loop (SIL) simulation is a cornerstone for the validation of modern automotive safety functions.<n>This paper proposes a perception-informed SIL testing methodology that bridges the gap between ground-truth simulation and real-world perception behavior.
  arXiv  Detail & Related papers  (2026-06-05T11:53:43Z)
• World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning [67.93052893044603]
  We study how a model learns to invoke, verify, and integrate visual future simulation alongside abstract reasoning.<n>PF-OPSD uses ground-truth future videos and answers only as teacher-side privileged context to evaluate on-policy concrete-reasoning trajectories.
  arXiv  Detail & Related papers  (2026-06-02T13:07:49Z)
• MANTA: Multi-turn Assessment for Nonhuman Thinking & Alignment [0.1452394725421793]
  MANTA is a dynamic multi-turn evaluation framework built on the Inspect AI platform.<n>We present preliminary results from evaluations of claude-sonnet-4-20250514 and openai/gpt-4o.<n>We additionally present STYLEJUDGE, a controlled four-judge study demonstrating systematic format bias in LLM-as-judge evaluation.
  arXiv  Detail & Related papers  (2026-04-18T19:51:32Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Models [8.630726904040781]
  Explicit Visual Premise Verification (EVPV) is a lightweight verification interface that conditions step scoring on the reliability of the visual premises a step depends on.<n>EVPV improves step-level verification and consistently boosts Best-of-N reranking accuracy over strong baselines.
  arXiv  Detail & Related papers  (2026-03-17T08:40:26Z)
• How Far Can Unsupervised RLVR Scale LLM Training? [57.44753418846446]
  Unsupervised reinforcement learning with verifiable rewards (URLVR) offers a pathway to scale LLM training beyond the supervision bottleneck.<n>Recent works leverage model intrinsic signals, showing promising early gains, yet their potential and limitations remain unclear.<n>We classify URLVR methods into intrinsic versus external based on reward sources, then establish a unified theoretical framework.
  arXiv  Detail & Related papers  (2026-03-09T17:38:11Z)
• The Shadow Self: Intrinsic Value Misalignment in Large Language Model Agents [37.75212140218036]
  We formalize the Loss-of-Control risk and identify the previously underexamined Intrinsic Value Misalignment (Intrinsic VM)<n>We then introduce IMPRESS, a scenario-driven framework for systematically assessing this risk.<n>We evaluate Intrinsic VM on 21 state-of-the-art LLM agents and find that it is a common and broadly observed safety risk across models.
  arXiv  Detail & Related papers  (2026-01-24T07:09:50Z)
• Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking [64.97768177044355]
  Large Language Models (LLMs) are increasingly deployed in real-world fact-checking systems.<n>We present FactArena, a fully automated arena-style evaluation framework.<n>Our analyses reveal significant discrepancies between static claim-verification accuracy and end-to-end fact-checking competence.
  arXiv  Detail & Related papers  (2026-01-06T02:51:56Z)
• Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning [71.19675094463834]
  This work introduces Counterfactual VLA, a self-reflective VLA framework that enables the model to reason about and revise its planned actions before execution.<n>CF-VLA first generates time-segmented meta-actions that summarize driving intent, and then performs counterfactual reasoning conditioned on both the meta-actions and the visual context.<n> Experiments on large-scale driving datasets show that CF-VLA improves trajectory accuracy by up to 17.6%, enhances safety metrics by 20.5%, and exhibits adaptive thinking.
  arXiv  Detail & Related papers  (2025-12-30T19:04:17Z)
• Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification [17.67273082468732]
  Verifiers -- functions assigning rewards to agent behavior -- have been key for AI progress in domains like math and board games.<n>We evaluate Multimodal Large Language Models (MLLMs) as verifiers of agent trajectories across web navigation, computer use, and robotic manipulation.<n>We propose Self-Grounded Verification (SGV), a lightweight method that enables more effective use of MLLMs' knowledge and reasoning.
  arXiv  Detail & Related papers  (2025-07-15T18:50:29Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.