SPHINX: First Explain, Then Explore

• URL: http://arxiv.org/abs/2606.17482v1
• Date: Tue, 16 Jun 2026 03:48:27 GMT
• Title: SPHINX: First Explain, Then Explore
• Abstract summary: We argue that adversarial scenes should be generated based on the failure diagnosis of the driving policy.<n>In this paper, we propose a closed-loop framework for adversarial scenario synthesis guided by a simple principle: first explain, then explore.
• Score: 9.461724985210312
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating adversarial driving scenarios is critical for evaluating and improving autonomous vehicle decision-making systems in simulation. Recent approaches, such as ChatScene and LLM-Attacker, rely primarily on the prior knowledge of Large Language Models and Vision-Language Models to generate driving scenarios procedurally. We argue that adversarial scenes should be generated based on the failure diagnosis (e.g., indecisiveness, multi-frame inconsistency) of the driving policy to specifically address the policy's weaknesses instead of relying on prior assumptions. In this paper, we propose SPHINX, a closed-loop framework for adversarial scenario synthesis guided by a simple principle: first explain, then explore. Beyond blindly exploring the scenario space, SPHINX leverages explainable artificial intelligence methods to analyze the policy, identifying key visual concepts and their influence on policy outputs, and the uncertainty of the decisions. Given the interpretable evidence extracted from the policy's own decision process, we use a vision language model to rationalize and criticize failure modes of the current policy. These critics are then used to generate targeted adversarial scenarios for policy retraining and improvement. We demonstrate that SPHINX can highlight an interpretable account of policy failures while other adversarial scene generation cannot. Across the evaluated benchmarks and test suites, SPHINX can be applied to diverse state-of-the-art autonomous vehicle architectures and yields consistent robustness improvements over existing scenario-generation methods.

Related papers

• Large Discrete Policy: Advancing Explicit Behavior Modeling with Stochastic Iterative Scoring [77.17706435067372]
  Large Policy (LDiP) is a fully discrete behavior modeling framework that selects actions from a large vocabulary of physically plausible candidates.<n>Rather than perturbing actions, LDiP improves expressivity through iterative scoring.<n>Across end-to-end planning, closed-loop driving, robotic manipulation, and vision-language-action settings, LDiP consistently outperforms strong discrete and continuous baselines.
  arXiv  Detail & Related papers  (2026-09-07T05:12:52Z)
• SPOTting the Future: Lookahead Explanations for Deep Reinforcement Learning [0.0]
  SPOT (Sampling Policy Observation Tree) is a model-agnostic, sampling-based framework for interpreting DRL policies.<n>We show how SPOT constructs an interpretable finite-horizon tree by sampling actions and simulating the resulting successor states.<n>The study illustrates how its tree-based representation can be used to inspect policy preferences, compare alternative future trajectories, and reveal downstream behaviors that are not visible through single-timestep feature-attribution methods.
  arXiv  Detail & Related papers  (2026-07-28T07:57:18Z)
• How Should World Models Be Evaluated? A Decision-Making-Centric Position [5.773605711982218]
  World models have rapidly become one of the central abstractions in modern AI.<n>Recent papers measure video realism, perceptual similarity, instruction following, physical plausibility, policy ranking, executability, planning success, and downstream policy improvement.<n>This paper argues that the central question is use-dependent.
  arXiv  Detail & Related papers  (2026-06-13T00:21:21Z)
• From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving [56.30087557121323]
  AlignADV is a learnability-guided closed-loop adversarial training framework.<n>It converts adversarial scenarios into resolvable and capability-aligned curricula.<n> Experiments show it reduces training steps by up to 40.6 percent.
  arXiv  Detail & Related papers  (2026-06-12T02:13:55Z)
• PolicyBank: Evolving Policy Understanding for LLM Agents [51.86716874651299]
  PolicyBank maintains structured, tool-level policy insights and iteratively refines them.<n>PolicyBank closes up to 82% of the gap toward a human oracle.
  arXiv  Detail & Related papers  (2026-04-16T20:29:30Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehicles [47.25901323750217]
  The generation of safety-critical scenarios in simulation has become increasingly crucial for safety evaluation in autonomous vehicles prior to road deployment in society.<n>We propose ScenGE, a framework that can generate plentiful safety-critical scenarios by reasoning novel adversarial cases and then amplifying them with complex traffic flows.<n>We validate our framework through real-world vehicle tests and human evaluation, confirming that the generated scenarios are both plausible and critical.
  arXiv  Detail & Related papers  (2025-08-20T08:36:57Z)
• From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment [11.799979691988902]
  FOREWARN is a novel framework to unlock the potential of Vision Language Models for runtime policy steering.<n>For foresight, we leverage a latent world model to imagine future latent states given diverse low-level action plans.<n>For forethought, we align the VLM with these predicted latent states to reason about the consequences of actions in its native representation.
  arXiv  Detail & Related papers  (2025-02-03T21:11:02Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.