Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing

• URL: http://arxiv.org/abs/2007.02787v1
• Date: Mon, 6 Jul 2020 14:42:11 GMT
• Title: Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing
• Authors: Vincenzo Riccio and Paolo Tonella
• Abstract summary: Deep Learning (DL) systems produce an output for any arbitrary numeric vector provided as input, regardless of whether it is within or outside the validity domain of the system under test. In this paper, we introduce the notion of frontier of behaviours, i.e., the inputs at which the DL system starts to misbehave. We developed DeepJanus, a search-based tool that generates frontier inputs for DL systems.
• Score: 4.632232395989182
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing adoption of Deep Learning (DL) for critical tasks, such as autonomous driving, the evaluation of the quality of systems that rely on DL has become crucial. Once trained, DL systems produce an output for any arbitrary numeric vector provided as input, regardless of whether it is within or outside the validity domain of the system under test. Hence, the quality of such systems is determined by the intersection between their validity domain and the regions where their outputs exhibit a misbehaviour. In this paper, we introduce the notion of frontier of behaviours, i.e., the inputs at which the DL system starts to misbehave. If the frontier of misbehaviours is outside the validity domain of the system, the quality check is passed. Otherwise, the inputs at the intersection represent quality deficiencies of the system. We developed DeepJanus, a search-based tool that generates frontier inputs for DL systems. The experimental results obtained for the lane keeping component of a self-driving car show that the frontier of a well trained system contains almost exclusively unrealistic roads that violate the best practices of civil engineering, while the frontier of a poorly trained one includes many valid inputs that point to serious deficiencies of the system.

Related papers

• Deep Learning System Boundary Testing through Latent Space Style Mixing [3.4561220135252277]
  We introduce MIMICRY, a novel black-box system-agnostic test generator to generate frontier inputs for the deep learning systems under test. MIMICRY uses style-based generative adversarial networks trained to learn the representation of inputs with disentangled features. We evaluated the effectiveness of different MIMICRY configurations in generating boundary inputs for four popular DL image classification systems.
  arXiv  Detail & Related papers  (2024-08-12T16:14:55Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems [0.7461036096470347]
  Traditional offline testing methods miss violation cases caused by dynamic objects like people and animals. Online testing methods require extensive training time, which is impractical with limited budgets. We introduce GARL, a framework combining a genetic algorithm (GA) and reinforcement learning (RL) for efficient generation of diverse and real landing system failures.
  arXiv  Detail & Related papers  (2023-10-11T10:54:01Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• Detecting and Mitigating System-Level Anomalies of Vision-Based Controllers [7.095058159492494]
  Vision-based controllers can make erroneous predictions when faced with novel or out-of-distribution inputs. In this work, we introduce a run-time anomaly monitor to detect and mitigate such closed-loop, system-level failures. We validate the proposed approach on an autonomous aircraft taxiing system that uses a vision-based controller for taxiing.
  arXiv  Detail & Related papers  (2023-09-23T20:33:38Z)
• Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding [51.8579160500354]
  We propose an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration. Results show equivalent or even more impressive performance compared to fully-supervised state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-03-17T00:28:33Z)
• Multimodal Detection of Unknown Objects on Roads for Autonomous Driving [4.3310896118860445]
  We propose a novel pipeline to detect unknown objects. We make use of lidar and camera data by combining state-of-the art detection models in a sequential manner.
  arXiv  Detail & Related papers  (2022-05-03T10:58:41Z)
• Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving [100.57791628642624]
  We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving. We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
  arXiv  Detail & Related papers  (2022-03-04T16:14:33Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)
• Out-of-Distribution Detection for Automotive Perception [58.34808836642603]
  Neural networks (NNs) are widely used for object classification in autonomous driving. NNs can fail on input data not well represented by the training dataset, known as out-of-distribution (OOD) data. This paper presents a method for determining whether inputs are OOD, which does not require OOD data during training and does not increase the computational cost of inference.
  arXiv  Detail & Related papers  (2020-11-03T01:46:35Z)
• Importance-Driven Deep Learning System Testing [12.483260526189449]
  Deep Learning (DL) systems are key enablers for engineering intelligent applications. Using DL systems in safety- and security-critical applications requires to provide testing evidence for their dependable operation. DeepImportance is a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion.
  arXiv  Detail & Related papers  (2020-02-09T19:20:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.