HABIT: Human Action Benchmark for Interactive Traffic in CARLA

• URL: http://arxiv.org/abs/2511.19109v1
• Date: Mon, 24 Nov 2025 13:43:39 GMT
• Title: HABIT: Human Action Benchmark for Interactive Traffic in CARLA
• Authors: Mohan Ramesh, Mark Azer, Fabian B. Flohr,
• Abstract summary: We introduce HABIT (Human Action Benchmark for Interactive Traffic), a high-fidelity simulation benchmark.<n>From an initial pool of approximately 30,000 retargeted motions, we curate 4,730 traffic-compatible pedestrian motions.<n>Our safety metrics, including Abbreviated Injury Scale (AIS) and False Positive Braking Rate (FPBR), reveal critical failure modes in state-of-the-art AD agents missed by prior evaluations.
• Score: 0.2905751301655124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current autonomous driving (AD) simulations are critically limited by their inadequate representation of realistic and diverse human behavior, which is essential for ensuring safety and reliability. Existing benchmarks often simplify pedestrian interactions, failing to capture complex, dynamic intentions and varied responses critical for robust system deployment. To overcome this, we introduce HABIT (Human Action Benchmark for Interactive Traffic), a high-fidelity simulation benchmark. HABIT integrates real-world human motion, sourced from mocap and videos, into CARLA (Car Learning to Act, a full autonomous driving simulator) via a modular, extensible, and physically consistent motion retargeting pipeline. From an initial pool of approximately 30,000 retargeted motions, we curate 4,730 traffic-compatible pedestrian motions, standardized in SMPL format for physically consistent trajectories. HABIT seamlessly integrates with CARLA's Leaderboard, enabling automated scenario generation and rigorous agent evaluation. Our safety metrics, including Abbreviated Injury Scale (AIS) and False Positive Braking Rate (FPBR), reveal critical failure modes in state-of-the-art AD agents missed by prior evaluations. Evaluating three state-of-the-art autonomous driving agents, InterFuser, TransFuser, and BEVDriver, demonstrates how HABIT exposes planner weaknesses that remain hidden in scripted simulations. Despite achieving close or equal to zero collisions per kilometer on the CARLA Leaderboard, the autonomous agents perform notably worse on HABIT, with up to 7.43 collisions/km and a 12.94% AIS 3+ injury risk, and they brake unnecessarily in up to 33% of cases. All components are publicly released to support reproducible, pedestrian-aware AI research.

Related papers

• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• SVBRD-LLM: Self-Verifying Behavioral Rule Discovery for Autonomous Vehicle Identification [1.6386429281694148]
  This paper proposes SVBRD-LLM, a framework that automatically discovers, verifies, and applies interpretable behavioral rules from real traffic videos.<n>The framework extracts vehicle trajectories using YOLOv8 and ByteTrack, computes kinematic features, and employs GPT-5 zero-shot prompting to compare autonomous and human-driven vehicles.<n> Experiments on over 1500 hours of real traffic videos show that the framework achieves 90.0% accuracy and 93.3% F1-score in autonomous vehicle identification.
  arXiv  Detail & Related papers  (2025-11-18T23:45:30Z)
• ZTRS: Zero-Imitation End-to-end Autonomous Driving with Trajectory Scoring [52.195295396336526]
  ZTRS (Zero-Imitation End-to-End Autonomous Driving with Trajectory Scoring) is a framework that combines the strengths of both worlds: sensor inputs without losing information and RL training for robust planning.<n>ZTRS demonstrates strong performance across three benchmarks: Navtest, Navhard, and HUGSIM.
  arXiv  Detail & Related papers  (2025-10-28T06:26:36Z)
• SPACeR: Self-Play Anchoring with Centralized Reference Models [50.55045557371374]
  Sim agent policies are realistic, human-like, fast, and scalable in multi-agent settings.<n>Recent progress in imitation learning with large diffusion-based or tokenized models has shown that behaviors can be captured directly from human driving data.<n>We propose SPACeR, a framework that leverages a pretrained tokenized autoregressive motion model as a central reference policy.
  arXiv  Detail & Related papers  (2025-10-20T19:53:02Z)
• DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning [59.63038625580992]
  Existing imitation learning approaches often fail to model realistic traffic behaviors.<n>We propose DecompGAIL, which explicitly decomposes realism into ego-map and ego-neighbor components.<n>DecompGAIL achieves state-of-the-art performance on the WOMD Sim Agents 2025 benchmark.
  arXiv  Detail & Related papers  (2025-10-08T11:46:39Z)
• MetAdv: A Unified and Interactive Adversarial Testing Platform for Autonomous Driving [85.04826012938642]
  MetAdv is a novel adversarial testing platform that enables realistic, dynamic, and interactive evaluation.<n>It supports flexible 3D vehicle modeling and seamless transitions between simulated and physical environments.<n>It enables real-time capture of physiological signals and behavioral feedback from drivers.
  arXiv  Detail & Related papers  (2025-08-04T03:07:54Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• Minds on the Move: Decoding Trajectory Prediction in Autonomous Driving with Cognitive Insights [18.92479778025183]
  In driving scenarios, a vehicle's trajectory is determined by the decision-making process of human drivers.<n>Previous models fail to capture the true intentions of human drivers, leading to suboptimal performance in long-term trajectory prediction.<n>We introduce a Cognitive-Informed Transformer (CITF) that incorporates a cognitive concept, Perceived Safety, to interpret drivers' decision-making mechanisms.
  arXiv  Detail & Related papers  (2025-02-27T13:43:17Z)
• Building reliable sim driving agents by scaling self-play [3.3378669626639423]
  Simulation agents are essential for designing and testing systems that interact with humans, such as autonomous vehicles (AVs)<n>We propose scaling self-play to thousands of scenarios on the Open Motion dataset under semi-realistic limits on human perception and control.<n>We generalize to unseen test scenes, achieving a 99.8% goal completion rate with less than 0.8% combined collision and off-road incidents.
  arXiv  Detail & Related papers  (2025-02-20T16:30:45Z)
• Are you a robot? Detecting Autonomous Vehicles from Behavior Analysis [6.422370188350147]
  We present a framework that monitors active vehicles using camera images and state information in order to determine whether vehicles are autonomous. Essentially, it builds on the cooperation among vehicles, which share their data acquired on the road feeding a machine learning model to identify autonomous cars. Experiments show it is possible to discriminate the two behaviors by analyzing video clips with an accuracy of 80%, which improves up to 93% when the target state information is available.
  arXiv  Detail & Related papers  (2024-03-14T17:00:29Z)
• What Matters to Enhance Traffic Rule Compliance of Imitation Learning for End-to-End Autonomous Driving [10.191916541924813]
  We proposed P-CSG, a penalty-based imitation learning approach with contrastive-based cross semantics generation sensor fusion technologies. In this paper, we introduce three penalties - red light, stop sign, and curvature speed penalty to make the agent more sensitive to traffic rules. We conducted robustness evaluations against adversarial attacks like FGSM and Dot attacks, revealing a substantial increase in robustness compared to other baseline models.
  arXiv  Detail & Related papers  (2023-09-14T15:54:56Z)
• ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events [1.84926694477846]
  We propose a black-box testing framework that uses offline trajectories first to analyze the existing behavior of autonomous vehicles. Our experiment shows an increase in 35, 23, 48, and 50% in the occurrences of vehicle collision, road object collision, pedestrian collision, and offroad steering events.
  arXiv  Detail & Related papers  (2023-08-28T13:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.