Towards a Trustworthy Anomaly Detection for Critical Applications through Approximated Partial AUC Loss

• URL: http://arxiv.org/abs/2502.11570v1
• Date: Mon, 17 Feb 2025 08:59:59 GMT
• Title: Towards a Trustworthy Anomaly Detection for Critical Applications through Approximated Partial AUC Loss
• Authors: Arnaud Bougaham, Benoît Frénay,
• Abstract summary: A binary classifier is trained to optimize the specific range of the AUC ROC curve that prevents the True Positive Rate (TPR) to reach 100% while minimizing the False Positive Rate (FPR) The results show a TPR of 92.52% at a 20.43% FPR for an average across 6 datasets, representing a TPR improvement of 4.3% for a FPR cost of 12.2% against other state-of-the-art methods.
• Score: 2.09942566943801
• License:
• Abstract: Anomaly Detection is a crucial step for critical applications such in the industrial, medical or cybersecurity domains. These sectors share the same requirement of handling differently the different types of classification errors. Indeed, even if false positives are acceptable, false negatives are not, because it would reflect a missed detection of a quality issue, a disease or a cyber threat. To fulfill this requirement, we propose a method that dynamically applies a trustworthy approximated partial AUC ROC loss (tapAUC). A binary classifier is trained to optimize the specific range of the AUC ROC curve that prevents the True Positive Rate (TPR) to reach 100% while minimizing the False Positive Rate (FPR). The optimal threshold that does not trigger any false negative is then kept and used at the test step. The results show a TPR of 92.52% at a 20.43% FPR for an average across 6 datasets, representing a TPR improvement of 4.3% for a FPR cost of 12.2% against other state-of-the-art methods. The code is available at https://github.com/ArnaudBougaham/tapAUC.

Related papers

• RadHop-Net: A Lightweight Radiomics-to-Error Regression for False Positive Reduction In MRI Prostate Cancer Detection [25.36983109088304]
  This paper introduces RadHop-Net, a novel and lightweight CNN for FP reduction. The proposed Stage 2 improves the average precision (AP) in lesion detection from 0.407 to 0.468 in the publicly available pi-cai dataset.
  arXiv  Detail & Related papers  (2025-01-03T19:20:04Z)
• FPR Estimation for Fraud Detection in the Presence of Class-Conditional Label Noise [0.0]
  We consider the problem of estimating the false-/ true-positive-rate (FPR/TPR) for a binary classification model when there are incorrect labels (label noise) in the validation set. Our motivating application is fraud prevention where accurate estimates of FPR are critical to preserving the experience for good customers.
  arXiv  Detail & Related papers  (2023-08-04T20:14:34Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• False: False Negative Samples Aware Contrastive Learning for Semantic Segmentation of High-Resolution Remote Sensing Image [11.356381535900901]
  We propose a False negAtive sampLes aware contraStive lEarning model (FALSE) for the semantic segmentation of high-resolution RSIs. Since the SSCL pretraining is unsupervised, the lack of definable criteria for false negative sample (FNS) leads to theoretical undecidability. We achieve coarse determination of FNS by the FNS self-determination strategy and achieve calibration of FNS by the FNS confidence calibration (FNCC) loss function.
  arXiv  Detail & Related papers  (2022-11-15T06:24:26Z)
• Large-scale Optimization of Partial AUC in a Range of False Positive Rates [51.12047280149546]
  The area under the ROC curve (AUC) is one of the most widely used performance measures for classification models in machine learning. We develop an efficient approximated gradient descent method based on recent practical envelope smoothing technique. Our proposed algorithm can also be used to minimize the sum of some ranked range loss, which also lacks efficient solvers.
  arXiv  Detail & Related papers  (2022-03-03T03:46:18Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Assessment of Treatment Effect Estimators for Heavy-Tailed Data [70.72363097550483]
  A central obstacle in the objective assessment of treatment effect (TE) estimators in randomized control trials (RCTs) is the lack of ground truth (or validation set) to test their performance. We provide a novel cross-validation-like methodology to address this challenge. We evaluate our methodology across 709 RCTs implemented in the Amazon supply chain.
  arXiv  Detail & Related papers  (2021-12-14T17:53:01Z)
• Leveraging Uncertainty for Improved Static Malware Detection Under Extreme False Positive Constraints [21.241478970181912]
  We show how ensembling and Bayesian treatments of machine learning methods for static malware detection allow for improved identification of model errors. In particular, we improve the true positive rate (TPR) at an actual realized FPR of 1e-5 from an expected 0.69 for previous methods to 0.80 on the best performing model class on the Sophos industry scale dataset.
  arXiv  Detail & Related papers  (2021-08-09T14:30:23Z)
• Adversarial Training with Rectified Rejection [114.83821848791206]
  We propose to use true confidence (T-Con) as a certainty oracle, and learn to predict T-Con by rectifying confidence. We prove that under mild conditions, a rectified confidence (R-Con) rejector and a confidence rejector can be coupled to distinguish any wrongly classified input from correctly classified ones.
  arXiv  Detail & Related papers  (2021-05-31T08:24:53Z)
• Constrained Optimization for Training Deep Neural Networks Under Class Imbalance [9.557146081524008]
  We introduce a novel constraint that can be used with existing loss functions to enforce maximal area under the ROC curve. We present experimental results for image-based classification applications using the CIFAR10 and an in-house medical imaging dataset.
  arXiv  Detail & Related papers  (2021-02-21T09:49:36Z)
• Second-Order Provable Defenses against Adversarial Attacks [63.34032156196848]
  We show that if the eigenvalues of the network are bounded, we can compute a certificate in the $l$ norm efficiently using convex optimization. We achieve certified accuracy of 5.78%, and 44.96%, and 43.19% on 2,59% and 4BP-based methods respectively.
  arXiv  Detail & Related papers  (2020-06-01T05:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.