Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge

• URL: http://arxiv.org/abs/2501.16037v2
• Date: Sat, 08 Feb 2025 00:11:43 GMT
• Title: Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge
• Authors: Anh-Kiet Duong, Petra Gomez-Krämer,
• Abstract summary: This paper presents a novel approach for hazard analysis in dashcam footage. It addresses the detection of driver reactions to hazards, the identification of hazardous objects, and the generation of descriptive captions. Our method achieved the highest scores in the Challenge on Out-of-Label in Autonomous Driving.
• Score: 0.0
• License:
• Abstract: This paper presents a novel approach for hazard analysis in dashcam footage, addressing the detection of driver reactions to hazards, the identification of hazardous objects, and the generation of descriptive captions. We first introduce a method for detecting driver reactions through speed and sound anomaly detection, leveraging unsupervised learning techniques. For hazard detection, we employ a set of heuristic rules as weak classifiers, which are combined using an ensemble method. This ensemble approach is further refined with differential privacy to mitigate overconfidence, ensuring robustness despite the lack of labeled data. Lastly, we use state-of-the-art vision-language models for hazard captioning, generating descriptive labels for the detected hazards. Our method achieved the highest scores in the Challenge on Out-of-Label in Autonomous Driving, demonstrating its effectiveness across all three tasks. Source codes are publicly available at https://github.com/ffyyytt/COOOL_2025.

Related papers

• Driver Assistance System Based on Multimodal Data Hazard Detection [0.0]
  This paper proposes a multimodal driver assistance detection system. It integrates road condition video, driver facial video, and audio data to enhance incident recognition accuracy.
  arXiv  Detail & Related papers  (2025-02-05T09:02:39Z)
• Zero-shot Hazard Identification in Autonomous Driving: A Case Study on the COOOL Benchmark [0.0]
  This paper presents our submission to the COOOL competition, a novel benchmark for detecting and classifying out-of-label hazards in autonomous driving. Our approach integrates diverse methods across three core tasks: (i) driver reaction detection, (ii) hazard object identification, and (iii) hazard captioning. The proposed pipeline outperformed the baseline methods by a large margin, reducing the relative error by 33%, and scored 2nd on the final leaderboard consisting of 32 teams.
  arXiv  Detail & Related papers  (2024-12-27T22:43:46Z)
• Semi-supervised Open-World Object Detection [74.95267079505145]
  We introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD) We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-02-25T07:12:51Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark [77.54411007883962]
  We propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames.
  arXiv  Detail & Related papers  (2022-12-19T11:43:02Z)
• Driving Anomaly Detection Using Conditional Generative Adversarial Network [26.45460503638333]
  This study proposes an unsupervised method to quantify driving anomalies using a conditional generative adversarial network (GAN) The approach predicts upcoming driving scenarios by conditioning the models on the previously observed signals. The results are validated with perceptual evaluations, where annotators are asked to assess the risk and familiarity of the videos detected with high anomaly scores.
  arXiv  Detail & Related papers  (2022-03-15T22:10:01Z)
• Sample-Efficient Safety Assurances using Conformal Prediction [57.92013073974406]
  Early warning systems can provide alerts when an unsafe situation is imminent. To reliably improve safety, these warning systems should have a provable false negative rate. We present a framework that combines a statistical inference technique known as conformal prediction with a simulator of robot/environment dynamics.
  arXiv  Detail & Related papers  (2021-09-28T23:00:30Z)
• Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection [55.12894776039135]
  State-of-the-art 3D object detectors, based on deep learning, have shown promising accuracy but are prone to over-fit to domain idiosyncrasies. We propose a novel learning approach that drastically reduces this gap by fine-tuning the detector on pseudo-labels in the target domain. We show, on five autonomous driving datasets, that fine-tuning the detector on these pseudo-labels substantially reduces the domain gap to new driving environments.
  arXiv  Detail & Related papers  (2021-03-26T01:18:11Z)
• Driver Anomaly Detection: A Dataset and Contrastive Learning Approach [17.020790792750457]
  We propose a contrastive learning approach to learn a metric to differentiate normal driving from anomalous driving. Our method reaches 0.9673 AUC on the test set, demonstrating the effectiveness of the contrastive learning approach on the anomaly detection task.
  arXiv  Detail & Related papers  (2020-09-30T13:23:21Z)
• Towards Anomaly Detection in Dashcam Videos [9.558392439655012]
  We propose to apply data-driven anomaly detection ideas from deep learning to dashcam videos. We present a large and diverse dataset of truck dashcam videos, namely RetroTrucks. We apply: (i) one-class classification loss and (ii) reconstruction-based loss, for anomaly detection on RetroTrucks.
  arXiv  Detail & Related papers  (2020-04-11T00:10:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.