PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment

• URL: http://arxiv.org/abs/2605.15654v1
• Date: Fri, 15 May 2026 06:20:43 GMT
• Title: PCASim: Promptable Closed-loop Adversarial Simulation for Urban Traffic Environment
• Abstract summary: A large language model (LLM) is employed to integrate knowledge-, data-, and adversarial-driven approaches.<n> reinforcement learning models are employed to train the behaviors of different types of vehicles.<n>Results demonstrate that the proposed framework improves the accuracy of domain-specific language generation by 12%.
• Score: 6.540774583094088
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world autonomous driving, particularly in urban environments with numerous corner cases, requires rigorous testing to ensure product safety and robustness. However, few studies have explored integrating adversarial scenario generation with the training of safety agents in closed-loop testing, enabling efficient co-evolution and mutual enhancement of both. To address this challenge, an adversarial behavior knowledge repository is constructed by applying rule-based filtering to an open-source dataset, combined with knowledge retrieval modules tailored for simulation environments. A large language model (LLM) is employed to integrate knowledge-, data-, and adversarial-driven approaches, generating safety-critical traffic scenarios customized to user needs. Additionally, while evaluating the generated scenarios, we employ reinforcement learning models to train the behaviors of different types of vehicles, thereby enriching scenario diversity beyond existing datasets while preserving realism. Experimental results demonstrate that the proposed framework improves the accuracy of domain-specific language generation by 12\%. Moreover, the success rate of newly generated scenario transformations increases by 8\%, while obstacle-avoidance capability is enhanced by 30\%. For the complete manuscript, please refer to: https://zhenhaooo.github.io/PCASim.github.io/

Related papers

• From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving [56.30087557121323]
  AlignADV is a learnability-guided closed-loop adversarial training framework.<n>It converts adversarial scenarios into resolvable and capability-aligned curricula.<n> Experiments show it reduces training steps by up to 40.6 percent.
  arXiv  Detail & Related papers  (2026-06-12T02:13:55Z)
• Generating Realistic Safety-Critical Scenarios for Vehicle-Pedestrian Interactions [3.8542226306135157]
  This study proposes a three-stage framework to generate behaviorally realistic safety-critical scenarios at scale.<n>Stage 1 pre-trains multi-agent state-space Transformer-enhanced DDPG agents on real-world safety-critical data to learn human-like interactive evasive behaviors.<n>Stage 2 deploys pre-trained multi-agents in CARLA for online reinforcement learning to generalize across diverse scenarios.<n>Stage 3 uses CARLA with the refined model to generate over 198,000 high-resolution interaction episodes from eight intersection scenarios.
  arXiv  Detail & Related papers  (2026-05-17T02:30:02Z)
• Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning [83.67663266193308]
  Class-Incremental Learning (CIL) requires a learning system to learn new classes while retaining previously learned knowledge.<n>In real-world scenarios such as autonomous driving, a system trained on urban roads in sunny weather may later need to operate in rural or highway environments.<n>We propose a unified framework to address domain shift and catastrophic forgetting simultaneously.
  arXiv  Detail & Related papers  (2026-05-09T09:49:13Z)
• Conditional Flow-VAE for Safety-Critical Traffic Scenario Generation [22.114606251423925]
  We introduce a conditional latent flow matching approach for scalable and realistic safety-critical scenario generation.<n>Our method uses distribution matching to transform nominal scenes into safety-critical rollouts.
  arXiv  Detail & Related papers  (2026-05-06T00:08:25Z)
• SG-CADVLM: A Context-Aware Decoding Powered Vision Language Model for Safety-Critical Scenario Generation [8.490714399662453]
  This paper presents SG-CADVLM, a framework that integrates Context-Aware Decoding with multi-modal input processing to generate safety-critical scenarios.<n>The framework mitigates VLM hallucination issues while enabling the simultaneous generation of road geometry and vehicle trajectories.
  arXiv  Detail & Related papers  (2026-01-26T12:53:12Z)
• From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios [6.681744368557208]
  Large Language Models (LLMs) and structured scenario parsing and prompt engineering are used to generate safety-critical driving scenarios.<n>We validate our approach using a 2D simulation framework and multiple pre-trained LLMs.<n>We conclude that an LLM equipped with domain-informed prompting techniques can effectively evaluate and generate safety-critical driving scenarios.
  arXiv  Detail & Related papers  (2025-02-04T09:19:13Z)
• ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable [88.08120417169971]
  Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner.
  arXiv  Detail & Related papers  (2024-09-12T08:26:33Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios [58.62407014256686]
  RealGen is a novel retrieval-based in-context learning framework for traffic scenario generation. RealGen synthesizes new scenarios by combining behaviors from multiple retrieved examples in a gradient-free way. This in-context learning framework endows versatile generative capabilities, including the ability to edit scenarios.
  arXiv  Detail & Related papers  (2023-12-19T23:11:06Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.