Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes

• URL: http://arxiv.org/abs/2309.04302v1
• Date: Fri, 8 Sep 2023 13:02:36 GMT
• Title: Have We Ever Encountered This Before? Retrieving Out-of-Distribution Road Obstacles from Driving Scenes
• Authors: Youssef Shoeb, Robin Chan, Gesina Schwalbe, Azarm Nowzard, Fatma G\"uney, Hanno Gottschalk
• Abstract summary: We propose a text-based approach to retrieve OoD road obstacles from driving scenes in video streams. Our proposed method leverages the recent advances in OoD segmentation and multi-modal foundation models. We present a first approach for the novel task of text-based OoD object retrieval.
• Score: 3.7748662901422807
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the life cycle of highly automated systems operating in an open and dynamic environment, the ability to adjust to emerging challenges is crucial. For systems integrating data-driven AI-based components, rapid responses to deployment issues require fast access to related data for testing and reconfiguration. In the context of automated driving, this especially applies to road obstacles that were not included in the training data, commonly referred to as out-of-distribution (OoD) road obstacles. Given the availability of large uncurated recordings of driving scenes, a pragmatic approach is to query a database to retrieve similar scenarios featuring the same safety concerns due to OoD road obstacles. In this work, we extend beyond identifying OoD road obstacles in video streams and offer a comprehensive approach to extract sequences of OoD road obstacles using text queries, thereby proposing a way of curating a collection of OoD data for subsequent analysis. Our proposed method leverages the recent advances in OoD segmentation and multi-modal foundation models to identify and efficiently extract safety-relevant scenes from unlabeled videos. We present a first approach for the novel task of text-based OoD object retrieval, which addresses the question ''Have we ever encountered this before?''.

Related papers

• Decentralized Semantic Traffic Control in AVs Using RL and DQN for Dynamic Roadblocks [9.485363025495225]
  We present a novel semantic traffic control system that entrusts semantic encoding responsibilities to the vehicles themselves. This system processes driving decisions obtained from a Reinforcement Learning (RL) agent, streamlining the decision-making process.
  arXiv  Detail & Related papers  (2024-06-26T20:12:48Z)
• RSRD: A Road Surface Reconstruction Dataset and Benchmark for Safe and Comfortable Autonomous Driving [67.09546127265034]
  Road surface reconstruction helps to enhance the analysis and prediction of vehicle responses for motion planning and control systems. We introduce the Road Surface Reconstruction dataset, a real-world, high-resolution, and high-precision dataset collected with a specialized platform in diverse driving conditions. It covers common road types containing approximately 16,000 pairs of stereo images, original point clouds, and ground-truth depth/disparity maps.
  arXiv  Detail & Related papers  (2023-10-03T17:59:32Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics [77.34726150561087]
  This work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. We give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.
  arXiv  Detail & Related papers  (2023-03-08T00:48:32Z)
• AI-Driven Road Maintenance Inspection v2: Reducing Data Dependency & Quantifying Road Damage [11.052502162865206]
  Road infrastructure maintenance inspection is typically a labor-intensive and critical task to ensure the safety of all road users. We propose an automated labelling pipeline by leveraging techniques like few-shot learning and out-of-distribution detection to generate labels for road damage detection. We show that the AI models trained with these techniques can not only generalize better to unseen real-world data with reduced requirement for human annotation but also provide an estimate of maintenance urgency.
  arXiv  Detail & Related papers  (2022-10-07T14:11:27Z)
• Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception [59.2014692323323]
  Small, far-away, or highly occluded objects are particularly challenging because there is limited information in the LiDAR point clouds for detecting them. We propose a novel, end-to-end trainable Hindsight framework to extract contextual information from past data. We show that this framework is compatible with most modern 3D detection architectures and can substantially improve their average precision on multiple autonomous driving datasets.
  arXiv  Detail & Related papers  (2022-03-22T00:58:27Z)
• Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review [4.228522109021283]
  We present a typical processing pipeline, which can be used to understand and interpret traffic videos. This processing framework includes several steps, including video enhancement, video stabilization, semantic and incident segmentation, object detection and classification, trajectory extraction, speed estimation, event analysis, modeling and anomaly detection.
  arXiv  Detail & Related papers  (2022-03-07T17:21:07Z)
• Unsupervised Lane-Change Identification for On-Ramp Merge Analysis in Naturalistic Driving Data [0.0]
  A scenario-driven approach has gained acceptance for CAVs emphasizing the requirement of a solid data basis of scenarios. This work proposes a framework for on-ramp scenario identification that also enables for scenario categorization and assessment. The efficacy of the framework is shown with a dataset collected on the Test Bed Lower Saxony.
  arXiv  Detail & Related papers  (2021-04-12T17:32:22Z)
• Batch Exploration with Examples for Scalable Robotic Reinforcement Learning [63.552788688544254]
  Batch Exploration with Examples (BEE) explores relevant regions of the state-space guided by a modest number of human provided images of important states. BEE is able to tackle challenging vision-based manipulation tasks both in simulation and on a real Franka robot.
  arXiv  Detail & Related papers  (2020-10-22T17:49:25Z)
• Towards Anomaly Detection in Dashcam Videos [9.558392439655012]
  We propose to apply data-driven anomaly detection ideas from deep learning to dashcam videos. We present a large and diverse dataset of truck dashcam videos, namely RetroTrucks. We apply: (i) one-class classification loss and (ii) reconstruction-based loss, for anomaly detection on RetroTrucks.
  arXiv  Detail & Related papers  (2020-04-11T00:10:40Z)
• Tracking Road Users using Constraint Programming [79.32806233778511]
  We present a constraint programming (CP) approach for the data association phase found in the tracking-by-detection paradigm of the multiple object tracking (MOT) problem. Our proposed method was tested on a motorized vehicles tracking dataset and produces results that outperform the top methods of the UA-DETRAC benchmark.
  arXiv  Detail & Related papers  (2020-03-10T00:04:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.