Corner case data description and detection
- URL: http://arxiv.org/abs/2101.02494v2
- Date: Fri, 12 Mar 2021 01:05:19 GMT
- Title: Corner case data description and detection
- Authors: Tinghui Ouyang, Vicent Sant Marco, Yoshinao Isobe, Hideki Asoh, Yutaka
Oiwa, Yoshiki Seo
- Abstract summary: Corner cases and related detection are crucial in AI quality assurance for constructing safety- and security-critical systems.
One is to enhance DL models robustness to corner case data via the adjustment on parameters/structure.
The other is to generate new corner cases for model retraining and improvement.
This paper proposes to a simple and novel study aiming at corner case data detection via a specific metric.
- Score: 3.4954795813842185
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: As the major factors affecting the safety of deep learning models, corner
cases and related detection are crucial in AI quality assurance for
constructing safety- and security-critical systems. The generic corner case
researches involve two interesting topics. One is to enhance DL models
robustness to corner case data via the adjustment on parameters/structure. The
other is to generate new corner cases for model retraining and improvement.
However, the complex architecture and the huge amount of parameters make the
robust adjustment of DL models not easy, meanwhile it is not possible to
generate all real-world corner cases for DL training. Therefore, this paper
proposes to a simple and novel study aiming at corner case data detection via a
specific metric. This metric is developed on surprise adequacy (SA) which has
advantages on capture data behaviors. Furthermore, targeting at characteristics
of corner case data, three modifications on distanced-based SA are developed
for classification applications in this paper. Consequently, through the
experiment analysis on MNIST data and industrial data, the feasibility and
usefulness of the proposed method on corner case data detection are verified.
Related papers
- Data Quality Issues in Vulnerability Detection Datasets [1.6114012813668932]
Vulnerability detection is a crucial yet challenging task to identify potential weaknesses in software for cyber security.
Deep learning (DL) has made great progress in automating the detection process.
Many datasets have been created to train DL models for this purpose.
However, these datasets suffer from several issues that will lead to low detection accuracy of DL models.
arXiv Detail & Related papers (2024-10-08T13:31:29Z) - Criteria for Uncertainty-based Corner Cases Detection in Instance Segmentation [2.9419365092937086]
We present corner case criteria based on the predictive uncertainty.
We evaluate each corner case criterion using the COCO and the NuImages dataset.
We also provide a corner case decision function that allows us to distinguish each object into True Positive (TP), localization and/or classification corner case, or False Positive (FP)
arXiv Detail & Related papers (2024-04-17T11:17:12Z) - Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification.
We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations.
Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
arXiv Detail & Related papers (2024-02-07T21:58:40Z) - Unsupervised Domain Adaptation for Self-Driving from Past Traversal
Features [69.47588461101925]
We propose a method to adapt 3D object detectors to new driving environments.
Our approach enhances LiDAR-based detection models using spatial quantized historical features.
Experiments on real-world datasets demonstrate significant improvements.
arXiv Detail & Related papers (2023-09-21T15:00:31Z) - CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs [6.131026007721575]
Corner case scenarios are an essential tool for testing and validating the safety of autonomous vehicles (AVs)
We introduce a novel approach based on Heterogeneous Graph Neural Networks (HGNNs) to transform regular driving scenarios into corner cases.
Our model successfully learned to produce corner cases from input scene graphs, achieving 89.9% prediction accuracy on our testing dataset.
arXiv Detail & Related papers (2023-09-18T14:59:11Z) - Enhancing Multiple Reliability Measures via Nuisance-extended
Information Bottleneck [77.37409441129995]
In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition.
We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training.
We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
arXiv Detail & Related papers (2023-03-24T16:03:21Z) - Online Safety Property Collection and Refinement for Safe Deep
Reinforcement Learning in Mapless Navigation [79.89605349842569]
We introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time.
CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties.
We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.
arXiv Detail & Related papers (2023-02-13T21:19:36Z) - Benchmarking the Robustness of LiDAR Semantic Segmentation Models [78.6597530416523]
In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions.
We propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy.
We design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications.
arXiv Detail & Related papers (2023-01-03T06:47:31Z) - Anomaly Detection Based on Selection and Weighting in Latent Space [73.01328671569759]
We propose a novel selection-and-weighting-based anomaly detection framework called SWAD.
Experiments on both benchmark and real-world datasets have shown the effectiveness and superiority of SWAD.
arXiv Detail & Related papers (2021-03-08T10:56:38Z) - An Application-Driven Conceptualization of Corner Cases for Perception
in Highly Automated Driving [21.67019631065338]
We provide an application-driven view of corner cases in highly automated driving.
We extend an existing camera-focused systematization of corner cases by adding RADAR and LiDAR.
We describe an exemplary toolchain for data acquisition and processing.
arXiv Detail & Related papers (2021-03-05T13:56:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.