A Trustworthiness Score to Evaluate DNN Predictions

• URL: http://arxiv.org/abs/2301.08839v6
• Date: Tue, 20 Jun 2023 14:52:47 GMT
• Title: A Trustworthiness Score to Evaluate DNN Predictions
• Authors: Abanoub Ghobrial, Darryl Hond, Hamid Asgari, Kerstin Eder
• Abstract summary: It is critical for safety during operation to know when deep neural networks' predictions are trustworthy or suspicious. We introduce the trustworthiness score (TS), a metric that provides a more transparent and effective way of providing confidence in predictions. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS.
• Score: 1.5484595752241122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the black box nature of deep neural networks (DNN), the continuous validation of DNN during operation is challenging with the absence of a human monitor. As a result this makes it difficult for developers and regulators to gain confidence in the deployment of autonomous systems employing DNN. It is critical for safety during operation to know when DNN's predictions are trustworthy or suspicious. With the absence of a human monitor, the basic approach is to use the model's output confidence score to assess if predictions are trustworthy or suspicious. However, the model's confidence score is a result of computations coming from a black box, therefore lacks transparency and makes it challenging to automatedly credit trustworthiness to predictions. We introduce the trustworthiness score (TS), a simple metric that provides a more transparent and effective way of providing confidence in DNN predictions compared to model's confidence score. The metric quantifies the trustworthiness in a prediction by checking for the existence of certain features in the predictions made by the DNN. We also use the underlying idea of the TS metric, to provide a suspiciousness score (SS) in the overall input frame to help in the detection of suspicious frames where false negatives exist. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS. The case study shows that using our method consistently improves the precision of predictions compared to relying on model confidence score alone, for both 1) approving of trustworthy predictions (~20% improvement) and 2) detecting suspicious frames (~5% improvement).

Related papers

• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• TrustGuard: GNN-based Robust and Explainable Trust Evaluation with Dynamicity Support [59.41529066449414]
  We propose TrustGuard, a GNN-based accurate trust evaluation model that supports trust dynamicity. TrustGuard is designed with a layered architecture that contains a snapshot input layer, a spatial aggregation layer, a temporal aggregation layer, and a prediction layer. Experiments show that TrustGuard outperforms state-of-the-art GNN-based trust evaluation models with respect to trust prediction across single-timeslot and multi-timeslot.
  arXiv  Detail & Related papers  (2023-06-23T07:39:12Z)
• Online Black-Box Confidence Estimation of Deep Neural Networks [0.0]
  We introduce the neighborhood confidence (NHC) which estimates the confidence of an arbitrary DNN for classification. The metric can be used for black-box systems since only the top-1 class output is required and does not need access to the gradients. Evaluation on different data distributions, including small in-domain distribution shifts, out-of-domain data or adversarial attacks, shows that the NHC performs better or on par with a comparable method for online white-box confidence estimation.
  arXiv  Detail & Related papers  (2023-02-27T08:30:46Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• TRUST-LAPSE: An Explainable and Actionable Mistrust Scoring Framework for Model Monitoring [4.262769931159288]
  We propose TRUST-LAPSE, a "mistrust" scoring framework for continuous model monitoring. We assess the trustworthiness of each input sample's model prediction using a sequence of latent-space embeddings. Our latent-space mistrust scores achieve state-of-the-art results with AUROCs of 84.1 (vision), 73.9 (audio), and 77.1 (clinical EEGs)
  arXiv  Detail & Related papers  (2022-07-22T18:32:38Z)
• TrustGNN: Graph Neural Network based Trust Evaluation via Learnable Propagative and Composable Nature [63.78619502896071]
  Trust evaluation is critical for many applications such as cyber security, social communication and recommender systems. We propose a new GNN based trust evaluation method named TrustGNN, which integrates smartly the propagative and composable nature of trust graphs. Specifically, TrustGNN designs specific propagative patterns for different propagative processes of trust, and distinguishes the contribution of different propagative processes to create new trust.
  arXiv  Detail & Related papers  (2022-05-25T13:57:03Z)
• Learning Confidence for Transformer-based Neural Machine Translation [38.679505127679846]
  We propose an unsupervised confidence estimate learning jointly with the training of the neural machine translation (NMT) model. We explain confidence as how many hints the NMT model needs to make a correct prediction, and more hints indicate low confidence. We demonstrate that our learned confidence estimate achieves high accuracy on extensive sentence/word-level quality estimation tasks.
  arXiv  Detail & Related papers  (2022-03-22T01:51:58Z)
• Learning to Predict Trustworthiness with Steep Slope Loss [69.40817968905495]
  We study the problem of predicting trustworthiness on real-world large-scale datasets. We observe that the trustworthiness predictors trained with prior-art loss functions are prone to view both correct predictions and incorrect predictions to be trustworthy. We propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other.
  arXiv  Detail & Related papers  (2021-09-30T19:19:09Z)
• An evaluation of word-level confidence estimation for end-to-end automatic speech recognition [70.61280174637913]
  We investigate confidence estimation for end-to-end automatic speech recognition (ASR) We provide an extensive benchmark of popular confidence methods on four well-known speech datasets. Our results suggest a strong baseline can be obtained by scaling the logits by a learnt temperature.
  arXiv  Detail & Related papers  (2021-01-14T09:51:59Z)
• PAC Confidence Predictions for Deep Neural Network Classifiers [28.61937254015157]
  Key challenge for deploying deep neural networks (DNNs) in safety critical settings is the need to provide rigorous ways to quantify their uncertainty. We propose an algorithm for constructing predicted classification confidences for DNNs that comes with provable correctness guarantees.
  arXiv  Detail & Related papers  (2020-11-02T04:09:17Z)
• Towards Time-Aware Context-Aware Deep Trust Prediction in Online Social Networks [0.4061135251278187]
  Trust can be defined as a measure to determine which source of information is reliable and with whom we should share or from whom we should accept information. There are several applications for trust in Online Social Networks (OSNs), including social spammer detection, fake news detection, retweet behaviour detection and recommender systems. Trust prediction is the process of predicting a new trust relation between two users who are not currently connected.
  arXiv  Detail & Related papers  (2020-03-21T01:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.