Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles

• URL: http://arxiv.org/abs/2410.15912v1
• Date: Mon, 21 Oct 2024 11:35:33 GMT
• Title: Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles
• Authors: Zhengming Wang, Junli Wang, Pengfei Li, Zhaohan Li, Peng Li, Yilun Chen,
• Abstract summary: We develop a benchmark for assessing motion planning capabilities in merging scenarios. Our approach involves other vehicles trained in large scale datasets with micro-behavioral characteristics. Extensive experiments have demonstrated the advanced nature of this evaluation benchmark.
• Score: 20.832829903505296
• License:
• Abstract: While the capabilities of autonomous driving have advanced rapidly, merging into dense traffic remains a significant challenge, many motion planning methods for this scenario have been proposed but it is hard to evaluate them. Most existing closed-loop simulators rely on rule-based controls for other vehicles, which results in a lack of diversity and randomness, thus failing to accurately assess the motion planning capabilities in highly interactive scenarios. Moreover, traditional evaluation metrics are insufficient for comprehensively evaluating the performance of merging in dense traffic. In response, we proposed a closed-loop evaluation benchmark for assessing motion planning capabilities in merging scenarios. Our approach involves other vehicles trained in large scale datasets with micro-behavioral characteristics that significantly enhance the complexity and diversity. Additionally, we have restructured the evaluation mechanism by leveraging large language models to assess each autonomous vehicle merging onto the main road. Extensive experiments have demonstrated the advanced nature of this evaluation benchmark. Through this benchmark, we have obtained an evaluation of existing methods and identified common issues. The environment and vehicle motion planning models we have designed can be accessed at https://anonymous.4open.science/r/Bench4Merge-EB5D

Related papers

• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [65.24988062003096]
  We present NAVSIM, a framework for benchmarking vision-based driving policies. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights.
  arXiv  Detail & Related papers  (2024-06-21T17:59:02Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving? [84.17711168595311]
  End-to-end autonomous driving has emerged as a promising research direction to target autonomy from a full-stack perspective. nuScenes dataset, characterized by relatively simple driving scenarios, leads to an under-utilization of perception information in end-to-end models. We introduce a new metric to evaluate whether the predicted trajectories adhere to the road.
  arXiv  Detail & Related papers  (2023-12-05T11:32:31Z)
• Interaction-Aware Decision-Making for Autonomous Vehicles in Forced Merging Scenario Leveraging Social Psychology Factors [7.812717451846781]
  We consider a behavioral model that incorporates both social behaviors and personal objectives of the interacting drivers. We develop a receding-horizon control-based decision-making strategy that estimates online the other drivers' intentions.
  arXiv  Detail & Related papers  (2023-09-25T19:49:14Z)
• Towards trustworthy multi-modal motion prediction: Holistic evaluation and interpretability of outputs [3.5240925434839054]
  We focus on evaluation criteria, robustness, and interpretability of outputs. We propose an intent prediction layer that can be attached to multi-modal motion prediction models.
  arXiv  Detail & Related papers  (2022-10-28T14:14:22Z)
• A Hierarchical Pedestrian Behavior Model to Generate Realistic Human Behavior in Traffic Simulation [11.525073205608681]
  We present a hierarchical pedestrian behavior model that generates high-level decisions through the use of behavior trees. A full implementation of our work is integrated into GeoScenario Server, a scenario definition and execution engine. Our model is shown to replicate the real-world pedestrians' trajectories with a high degree of fidelity and a decision-making accuracy of 98% or better.
  arXiv  Detail & Related papers  (2022-06-01T02:04:38Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Deep Structured Reactive Planning [94.92994828905984]
  We propose a novel data-driven, reactive planning objective for self-driving vehicles. We show that our model outperforms a non-reactive variant in successfully completing highly complex maneuvers.
  arXiv  Detail & Related papers  (2021-01-18T01:43:36Z)
• Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarios [10.53961877853783]
  We propose an adaptive evaluation framework to efficiently evaluate autonomous vehicles in adversarial environments. Considering the multimodal nature of dangerous scenarios, we use ensemble models to represent different local optimums for diversity. Results show that the adversarial scenarios generated by our method significantly degrade the performance of the tested vehicles.
  arXiv  Detail & Related papers  (2020-04-14T14:12:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.