Statistical Test for Anomaly Detections by Variational Auto-Encoders

• URL: http://arxiv.org/abs/2402.03724v2
• Date: Sun, 2 Jun 2024 07:11:28 GMT
• Title: Statistical Test for Anomaly Detections by Variational Auto-Encoders
• Authors: Daiki Miwa, Tomohiro Shiraishi, Vo Nguyen Le Duy, Teruyuki Katsuoka, Ichiro Takeuchi,
• Abstract summary: We consider the reliability assessment of anomaly detection using Variational Autoencoder (VAE) Using the VAE-AD Test, the reliability of the anomaly regions detected by a VAE can be quantified in the form of p-values.
• Score: 19.927066428010782
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we consider the reliability assessment of anomaly detection (AD) using Variational Autoencoder (VAE). Over the last decade, VAE-based AD has been actively studied in various perspective, from method development to applied research. However, when the results of ADs are used in high-stakes decision-making, such as in medical diagnosis, it is necessary to ensure the reliability of the detected anomalies. In this study, we propose the VAE-AD Test as a method for quantifying the statistical reliability of VAE-based AD within the framework of statistical testing. Using the VAE-AD Test, the reliability of the anomaly regions detected by a VAE can be quantified in the form of p-values. This means that if an anomaly is declared when the p-value is below a certain threshold, it is possible to control the probability of false detection to a desired level. Since the VAE-AD Test is constructed based on a new statistical inference framework called selective inference, its validity is theoretically guaranteed in finite samples. To demonstrate the validity and effectiveness of the proposed VAE-AD Test, numerical experiments on artificial data and applications to brain image analysis are conducted.

Related papers

• Internal Incoherency Scores for Constraint-based Causal Discovery Algorithms [12.524536193679124]
  We propose internal coherency scores that allow testing for assumption violations and finite sample errors. We illustrate our coherency scores on the PC algorithm with simulated and real-world datasets.
  arXiv  Detail & Related papers  (2025-02-20T16:44:54Z)
• Statistically Significant $k$NNAD by Selective Inference [12.703556860454565]
  A critical challenge in anomaly detection, including kNNAD, is appropriately quantifying the reliability of detected anomalies. We formulate kNNAD as a statistical hypothesis test and quantify the probability of false detection using $p$-values. By leveraging SI, the Stat-kNNAD method ensures that detected anomalies are statistically significant with theoretical guarantees.
  arXiv  Detail & Related papers  (2025-02-18T15:58:58Z)
• Testing Conditional Mean Independence Using Generative Neural Networks [8.323172773256449]
  We introduce a novel population CMI measure and a bootstrap model-based testing procedure. Deep generative neural networks are used to estimate the conditional mean functions involved in the population measure.
  arXiv  Detail & Related papers  (2025-01-28T23:35:24Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Statistical Test on Diffusion Model-based Anomaly Detection by Selective Inference [19.927066428010782]
  We address the task of detecting anomalous regions in medical images using diffusion models. We propose a statistical method to quantify the reliability of the detected anomalies.
  arXiv  Detail & Related papers  (2024-02-19T02:32:45Z)
• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• From Static Benchmarks to Adaptive Testing: Psychometrics in AI Evaluation [60.14902811624433]
  We discuss a paradigm shift from static evaluation methods to adaptive testing. This involves estimating the characteristics and value of each test item in the benchmark and dynamically adjusting items in real-time. We analyze the current approaches, advantages, and underlying reasons for adopting psychometrics in AI evaluation.
  arXiv  Detail & Related papers  (2023-06-18T09:54:33Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Towards Reliable Medical Image Segmentation by utilizing Evidential Calibrated Uncertainty [52.03490691733464]
  We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks. By leveraging subjective logic theory, we explicitly model probability and uncertainty for the problem of medical image segmentation. DeviS incorporates an uncertainty-aware filtering module, which utilizes the metric of uncertainty-calibrated error to filter reliable data.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Adaptive novelty detection with false discovery rate guarantee [1.8249324194382757]
  We propose a flexible method to control the false discovery rate (FDR) on detected novelties in finite samples. Inspired by the multiple testing literature, we propose variants of AdaDetect that are adaptive to the proportion of nulls. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.
  arXiv  Detail & Related papers  (2022-08-13T17:14:55Z)
• Density of States Estimation for Out-of-Distribution Detection [69.90130863160384]
  DoSE is the density of states estimator. We demonstrate DoSE's state-of-the-art performance against other unsupervised OOD detectors.
  arXiv  Detail & Related papers  (2020-06-16T16:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.