Statistical Test for Generated Hypotheses by Diffusion Models
- URL: http://arxiv.org/abs/2402.11789v1
- Date: Mon, 19 Feb 2024 02:32:45 GMT
- Title: Statistical Test for Generated Hypotheses by Diffusion Models
- Authors: Teruyuki Katsuoka, Tomohiro Shiraishi, Daiki Miwa, Vo Nguyen Le Duy,
Ichiro Takeuchi
- Abstract summary: We consider a medical diagnostic task using generated images by diffusion models, and propose a statistical test to quantify its reliability.
Using the proposed method, the statistical reliability of medical image diagnostic results can be quantified in the form of a p-value, allowing for decision-making with a controlled error rate.
- Score: 21.378672594642616
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The enhanced performance of AI has accelerated its integration into
scientific research. In particular, the use of generative AI to create
scientific hypotheses is promising and is increasingly being applied across
various fields. However, when employing AI-generated hypotheses for critical
decisions, such as medical diagnoses, verifying their reliability is crucial.
In this study, we consider a medical diagnostic task using generated images by
diffusion models, and propose a statistical test to quantify its reliability.
The basic idea behind the proposed statistical test is to employ a selective
inference framework, where we consider a statistical test conditional on the
fact that the generated images are produced by a trained diffusion model. Using
the proposed method, the statistical reliability of medical image diagnostic
results can be quantified in the form of a p-value, allowing for
decision-making with a controlled error rate. We show the theoretical validity
of the proposed statistical test and its effectiveness through numerical
experiments on synthetic and brain image datasets.
Related papers
- Unified Uncertainty Estimation for Cognitive Diagnosis Models [70.46998436898205]
We propose a unified uncertainty estimation approach for a wide range of cognitive diagnosis models.
We decompose the uncertainty of diagnostic parameters into data aspect and model aspect.
Our method is effective and can provide useful insights into the uncertainty of cognitive diagnosis.
arXiv Detail & Related papers (2024-03-09T13:48:20Z) - Statistical Test for Anomaly Detections by Variational Auto-Encoders [19.927066428010782]
We consider the reliability assessment of anomaly detection using Variational Autoencoder (VAE)
Using the VAE-AD Test, the reliability of the anomaly regions detected by a VAE can be quantified in the form of p-values.
arXiv Detail & Related papers (2024-02-06T05:42:27Z) - MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data
Augmentation [58.93221876843639]
This paper introduces a novel, end-to-end diffusion-based risk prediction model, named MedDiffusion.
It enhances risk prediction performance by creating synthetic patient data during training to enlarge sample space.
It discerns hidden relationships between patient visits using a step-wise attention mechanism, enabling the model to automatically retain the most vital information for generating high-quality data.
arXiv Detail & Related papers (2023-10-04T01:36:30Z) - Statistical Hypothesis Testing for Information Value (IV) [0.0]
We propose a non-parametric hypothesis test to evaluate the predictive power of features contemplated in a data set.
We show how to efficiently compute our test statistic and we study its performance on simulated data.
arXiv Detail & Related papers (2023-09-22T21:16:18Z) - Synthetic Power Analyses: Empirical Evaluation and Application to
Cognitive Neuroimaging [14.57108653193695]
We propose a framework for estimating statistical power at various sample sizes.
We empirically explore the performance of synthetic power analysis for sample size selection in cognitive neuroscience experiments.
Our empirical results suggest that synthetic power analysis could be a low-cost alternative to pilot data collection.
arXiv Detail & Related papers (2022-10-11T23:33:32Z) - Statistical quantification of confounding bias in predictive modelling [0.0]
I propose the partial and full confounder tests, which probe the null hypotheses of unconfounded and fully confounded models.
The tests provide a strict control for Type I errors and high statistical power, even for non-normally and non-linearly dependent predictions.
arXiv Detail & Related papers (2021-11-01T10:35:24Z) - UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced
Data [81.00385374948125]
We present UNcertaInTy-based hEalth risk prediction (UNITE) model.
UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data.
We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD)
UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
arXiv Detail & Related papers (2020-10-22T02:28:11Z) - Balance-Subsampled Stable Prediction [55.13512328954456]
We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design.
A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift.
Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
arXiv Detail & Related papers (2020-06-08T07:01:38Z) - Semi-supervised Medical Image Classification with Relation-driven
Self-ensembling Model [71.80319052891817]
We present a relation-driven semi-supervised framework for medical image classification.
It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations.
Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
arXiv Detail & Related papers (2020-05-15T06:57:54Z) - Hemogram Data as a Tool for Decision-making in COVID-19 Management:
Applications to Resource Scarcity Scenarios [62.997667081978825]
COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure.
This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients.
Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
arXiv Detail & Related papers (2020-05-10T01:45:03Z) - Statistical Agnostic Mapping: a Framework in Neuroimaging based on
Concentration Inequalities [0.0]
We derive a Statistical Agnostic (non-parametric) Mapping at voxel or multi-voxel level.
We propose a novel framework in neuroimaging based on concentration inequalities.
arXiv Detail & Related papers (2019-12-27T18:27:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.