Zero-shot Hazard Identification in Autonomous Driving: A Case Study on the COOOL Benchmark

• URL: http://arxiv.org/abs/2412.19944v1
• Date: Fri, 27 Dec 2024 22:43:46 GMT
• Title: Zero-shot Hazard Identification in Autonomous Driving: A Case Study on the COOOL Benchmark
• Authors: Lukas Picek, Vojtěch Čermák, Marek Hanzl,
• Abstract summary: This paper presents our submission to the COOOL competition, a novel benchmark for detecting and classifying out-of-label hazards in autonomous driving. Our approach integrates diverse methods across three core tasks: (i) driver reaction detection, (ii) hazard object identification, and (iii) hazard captioning. The proposed pipeline outperformed the baseline methods by a large margin, reducing the relative error by 33%, and scored 2nd on the final leaderboard consisting of 32 teams.
• Score: 0.0
• License:
• Abstract: This paper presents our submission to the COOOL competition, a novel benchmark for detecting and classifying out-of-label hazards in autonomous driving. Our approach integrates diverse methods across three core tasks: (i) driver reaction detection, (ii) hazard object identification, and (iii) hazard captioning. We propose kernel-based change point detection on bounding boxes and optical flow dynamics for driver reaction detection to analyze motion patterns. For hazard identification, we combined a naive proximity-based strategy with object classification using a pre-trained ViT model. At last, for hazard captioning, we used the MOLMO vision-language model with tailored prompts to generate precise and context-aware descriptions of rare and low-resolution hazards. The proposed pipeline outperformed the baseline methods by a large margin, reducing the relative error by 33%, and scored 2nd on the final leaderboard consisting of 32 teams.

Related papers

• Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge [0.0]
  This paper presents a novel approach for hazard analysis in dashcam footage. It addresses the detection of driver reactions to hazards, the identification of hazardous objects, and the generation of descriptive captions. Our method achieved the highest scores in the Challenge on Out-of-Label in Autonomous Driving.
  arXiv  Detail & Related papers  (2025-01-27T13:32:01Z)
• A New Adversarial Perspective for LiDAR-based 3D Object Detection [15.429996348453967]
  We introduce a real-world dataset (ROLiD) comprising LiDAR-scanned point clouds of two random objects: water mist and smoke. We propose a point cloud sequence generation method using a motion and content decomposition generative adversarial network named PCS-GAN. Experiments demonstrate that adversarial perturbations based on random objects effectively deceive vehicle detection and reduce the recognition rate of 3D object detection models.
  arXiv  Detail & Related papers  (2024-12-17T15:36:55Z)
• Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning [13.613407983544427]
  We introduce a robust model designed to withstand changes in camera position within the vehicle. Our Driver Behavior Monitoring Network (DBMNet) relies on a lightweight backbone and integrates a disentanglement module. Experiments conducted on the daytime and nighttime subsets of the 100-Driver dataset validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-11-20T10:27:12Z)
• FuzzRisk: Online Collision Risk Estimation for Autonomous Vehicles based on Depth-Aware Object Detection via Fuzzy Inference [6.856508678236828]
  The framework takes two sets of predictions from different algorithms and associates their inconsistencies with the collision risk via fuzzy inference. We experimentally validate that, based on Intersection-over-Union (IoU) and a depth discrepancy measure, the inconsistencies between the two sets of predictions strongly correlate to the error of the 3D object detector. We optimize the fuzzy inference system towards an existing offline metric that matches AV collision rates well.
  arXiv  Detail & Related papers  (2024-11-09T20:20:36Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Probabilistic Risk Assessment of an Obstacle Detection System for GoA 4 Freight Trains [0.0]
  In this paper, a quantitative risk assessment approach is discussed for the design of an obstacle detection function for low-speed freight trains. It is illustrated that, under certain not unreasonable assumptions, the resulting hazard rate becomes acceptable for specific application settings. The statistical approach for assessing the residual risk of misclassifications in convolutional neural networks and conventional image processing software suggests that high confidence can be placed into the safety-critical obstacle detection function.
  arXiv  Detail & Related papers  (2023-06-26T16:18:20Z)
• Exploring Active 3D Object Detection from a Generalization Perspective [58.597942380989245]
  Uncertainty-based active learning policies fail to balance the trade-off between point cloud informativeness and box-level annotation costs. We propose textscCrb, which hierarchically filters out the point clouds of redundant 3D bounding box labels. Experiments show that the proposed approach outperforms existing active learning strategies.
  arXiv  Detail & Related papers  (2023-01-23T02:43:03Z)
• GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimation [70.75100533512021]
  In this paper, we formulate the label uncertainty problem as the diversity of potentially plausible bounding boxes of objects. We propose GLENet, a generative framework adapted from conditional variational autoencoders, to model the one-to-many relationship between a typical 3D object and its potential ground-truth bounding boxes with latent variables. The label uncertainty generated by GLENet is a plug-and-play module and can be conveniently integrated into existing deep 3D detectors.
  arXiv  Detail & Related papers  (2022-07-06T06:26:17Z)
• OW-DETR: Open-world Detection Transformer [90.56239673123804]
  We introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection. OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification and objectness scoring. Our model outperforms the recently introduced OWOD approach, ORE, with absolute gains ranging from 1.8% to 3.3% in terms of unknown recall.
  arXiv  Detail & Related papers  (2021-12-02T18:58:30Z)
• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [104.04999499189402]
  Out-of-training-distribution (OOD) scenarios are a common challenge of learning agents at deployment. We propose an uncertainty-aware planning method, called emphrobust imitative planning (RIP) Our method can detect and recover from some distribution shifts, reducing the overconfident and catastrophic extrapolations in OOD scenes. We introduce an autonomous car novel-scene benchmark, textttCARNOVEL, to evaluate the robustness of driving agents to a suite of tasks with distribution shifts.
  arXiv  Detail & Related papers  (2020-06-26T11:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.