Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic

• URL: http://arxiv.org/abs/2206.14372v2
• Date: Tue, 21 Nov 2023 07:22:04 GMT
• Title: Formalizing and Evaluating Requirements of Perception Systems for Automated Vehicles using Spatio-Temporal Perception Logic
• Authors: Mohammad Hekmatnejad, Bardh Hoxha, Jyotirmoy V. Deshmukh, Yezhou Yang, and Georgios Fainekos
• Abstract summary: We present a logic that enables reasoning over perception data using spatial and temporal operators. One major advantage ofSTPL is that it facilitates basic sanity checks on the functional performance of the perception system.
• Score: 25.070876549371693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated vehicles (AV) heavily depend on robust perception systems. Current methods for evaluating vision systems focus mainly on frame-by-frame performance. Such evaluation methods appear to be inadequate in assessing the performance of a perception subsystem when used within an AV. In this paper, we present a logic -- referred to as Spatio-Temporal Perception Logic (STPL) -- which utilizes both spatial and temporal modalities. STPL enables reasoning over perception data using spatial and temporal operators. One major advantage of STPL is that it facilitates basic sanity checks on the functional performance of the perception system, even without ground-truth data in some cases. We identify a fragment of STPL which is efficiently monitorable offline in polynomial time. Finally, we present a range of specifications for AV perception systems to highlight the types of requirements that can be expressed and analyzed through offline monitoring with STPL.

Related papers

• End-to-end Evaluation of Practical Video Analytics Systems for Face Detection and Recognition [9.942007083253479]
  Video analytics systems are deployed in bandwidth constrained environments like autonomous vehicles. In an end-to-end face analytics system, inputs are first compressed using popular video codecs like HEVC. We demonstrate how independent task evaluations, dataset imbalances, and inconsistent annotations can lead to incorrect system performance estimates.
  arXiv  Detail & Related papers  (2023-10-10T19:06:10Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States [84.24300005271185]
  We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
  arXiv  Detail & Related papers  (2023-01-27T22:28:19Z)
• Free Lunch for Generating Effective Outlier Supervision [46.37464572099351]
  We propose an ultra-effective method to generate near-realistic outlier supervision. Our proposed textttBayesAug significantly reduces the false positive rate over 12.50% compared with the previous schemes.
  arXiv  Detail & Related papers  (2023-01-17T01:46:45Z)
• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [132.20119288212376]
  We propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously. To the best of our knowledge, we are the first to systematically investigate each part of an interpretable end-to-end vision-based autonomous driving system.
  arXiv  Detail & Related papers  (2022-07-15T16:57:43Z)
• Efficient Human Vision Inspired Action Recognition using Adaptive Spatiotemporal Sampling [13.427887784558168]
  We introduce a novel adaptive vision system for efficient action recognition processing. Our system pre-scans the global context sampling scheme at low-resolution and decides to skip or request high-resolution features at salient regions for further processing. We validate the system on EPIC-KENS and UCF-101 datasets for action recognition, and show that our proposed approach can greatly speed up inference with a tolerable loss of accuracy compared with those from state-the-art baselines.
  arXiv  Detail & Related papers  (2022-07-12T01:18:58Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)
• A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control [14.713547378267748]
  We present a state-of-the-art reinforcement learning method for autonomous driving. We trained our system using both ground truth and estimated semantic segmentation input. The required training time of the system is shown to be lower and the performance on the benchmark superior to competing approaches.
  arXiv  Detail & Related papers  (2021-04-08T14:34:57Z)
• Monitoring and Diagnosability of Perception Systems [21.25149064251918]
  We propose a mathematical model for runtime monitoring and fault detection and identification in perception systems. We demonstrate our monitoring system, dubbed PerSyS, in realistic simulations using the LGSVL self-driving simulator and the Apollo Auto autonomy software stack.
  arXiv  Detail & Related papers  (2020-11-11T23:03:14Z)
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization [69.12982544509427]
  S-temporal action localization is an important problem in computer vision. We propose an end-to-end unsupervised domain adaptation algorithm. We show that significant performance gain can be achieved when spatial and temporal features are adapted separately or jointly.
  arXiv  Detail & Related papers  (2020-10-19T04:25:10Z)
• Monitoring and Diagnosability of Perception Systems [21.25149064251918]
  Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving cars. Despite the paramount importance of perception systems, there is no formal approach for system-level monitoring. We propose a mathematical model for runtime monitoring and fault detection of perception systems.
  arXiv  Detail & Related papers  (2020-05-24T18:09:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.