Virtual imaging trials improved the transparency and reliability of AI systems in COVID-19 imaging
- URL: http://arxiv.org/abs/2308.09730v2
- Date: Sun, 31 Mar 2024 19:28:25 GMT
- Title: Virtual imaging trials improved the transparency and reliability of AI systems in COVID-19 imaging
- Authors: Fakrul Islam Tushar, Lavsen Dahal, Saman Sotoudeh-Paima, Ehsan Abadi, W. Paul Segars, Ehsan Samei, Joseph Y. Lo,
- Abstract summary: The credibility of AI models in medical imaging is often challenged by issues and obscured clinical insights.
We propose a virtual imaging trial framework, employing a diverse collection of medical images that are both clinical and simulated.
- Score: 1.6040478776985583
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: The credibility of AI models in medical imaging is often challenged by reproducibility issues and obscured clinical insights, a reality highlighted during the COVID-19 pandemic by many reports of near-perfect artificial intelligence (AI) models that all failed to generalize. To address these concerns, we propose a virtual imaging trial framework, employing a diverse collection of medical images that are both clinical and simulated. In this study, COVID-19 serves as a case example to unveil the intrinsic and extrinsic factors influencing AI performance. Our findings underscore a significant impact of dataset characteristics on AI efficacy. Even when trained on large, diverse clinical datasets with thousands of patients, AI performance plummeted by up to 20% in generalization. However, virtual imaging trials offer a robust platform for objective assessment, unveiling nuanced insights into the relationships between patient- and physics-based factors and AI performance. For instance, disease extent markedly influenced AI efficacy, computed tomography (CT) out-performed chest radiography (CXR), while imaging dose exhibited minimal impact. Using COVID-19 as a case study, this virtual imaging trial study verified that radiology AI models often suffer from a reproducibility crisis. Virtual imaging trials not only offered a solution for objective performance assessment but also extracted several clinical insights. This study illuminates the path for leveraging virtual imaging to augment the reliability, transparency, and clinical relevance of AI in medical imaging.
Related papers
- Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging [2.0890189482817165]
We introduce a novel analysis framework for investigating the impact of biases in medical images on AI models.
We developed and tested this framework for conducting controlled in silico trials to assess bias in medical imaging AI.
arXiv Detail & Related papers (2023-11-03T01:37:28Z) - Knowledge-based in silico models and dataset for the comparative
evaluation of mammography AI for a range of breast characteristics, lesion
conspicuities and doses [2.9362519537872647]
We release M-SYNTH, a dataset of cohorts with four breast fibroglandular density distributions imaged at different exposure levels.
We find that model performance decreases with increasing breast density and increases with higher mass density, as expected.
As exposure levels decrease, AI model performance drops with the highest performance achieved at exposure levels lower than the nominal recommended dose for the breast type.
arXiv Detail & Related papers (2023-10-27T21:14:30Z) - Optimising Chest X-Rays for Image Analysis by Identifying and Removing
Confounding Factors [49.005337470305584]
During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions.
The variable quality of clinically-acquired CXRs within publicly available datasets could have a profound effect on algorithm performance.
We propose a simple and effective step-wise approach to pre-processing a COVID-19 chest X-ray dataset to remove undesired biases.
arXiv Detail & Related papers (2022-08-22T13:57:04Z) - SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In
Silico Experiments [12.019996672009375]
We show that creating realistic simulated images from human models is a viable alternative to large-scale in situ data collection.
Because synthetic generation of training data from human-based models scales easily, we find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real data-trained models.
arXiv Detail & Related papers (2022-06-13T13:08:41Z) - Robust and Efficient Medical Imaging with Self-Supervision [80.62711706785834]
We present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI.
We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data.
arXiv Detail & Related papers (2022-05-19T17:34:18Z) - Virtual vs. Reality: External Validation of COVID-19 Classifiers using
XCAT Phantoms for Chest Computed Tomography [2.924350993741562]
We created the CVIT-COVID dataset including 180 virtually imaged computed tomography (CT) images from simulated COVID-19 and normal phantom models.
We evaluated the performance of an open-source, deep-learning model from the University of Waterloo trained with multi-institutional data.
We validated the model's performance against open clinical data of 305 CT images to understand virtual vs. real clinical data performance.
arXiv Detail & Related papers (2022-03-07T00:11:53Z) - COVID-Net US: A Tailored, Highly Efficient, Self-Attention Deep
Convolutional Neural Network Design for Detection of COVID-19 Patient Cases
from Point-of-care Ultrasound Imaging [101.27276001592101]
We introduce COVID-Net US, a highly efficient, self-attention deep convolutional neural network design tailored for COVID-19 screening from lung POCUS images.
Experimental results show that the proposed COVID-Net US can achieve an AUC of over 0.98 while achieving 353X lower architectural complexity, 62X lower computational complexity, and 14.3X faster inference times on a Raspberry Pi.
To advocate affordable healthcare and artificial intelligence for resource-constrained environments, we have made COVID-Net US open source and publicly available as part of the COVID-Net open source initiative.
arXiv Detail & Related papers (2021-08-05T16:47:33Z) - COVIDx-US -- An open-access benchmark dataset of ultrasound imaging data
for AI-driven COVID-19 analytics [116.6248556979572]
COVIDx-US is an open-access benchmark dataset of COVID-19 related ultrasound imaging data.
It consists of 93 lung ultrasound videos and 10,774 processed images of patients infected with SARS-CoV-2 pneumonia, non-SARS-CoV-2 pneumonia, as well as healthy control cases.
arXiv Detail & Related papers (2021-03-18T03:31:33Z) - Integrative Analysis for COVID-19 Patient Outcome Prediction [53.11258640541513]
We combine radiomics of lung opacities and non-imaging features from demographic data, vital signs, and laboratory findings to predict need for intensive care unit admission.
Our methods may also be applied to other lung diseases including but not limited to community acquired pneumonia.
arXiv Detail & Related papers (2020-07-20T19:08:50Z) - Review of Artificial Intelligence Techniques in Imaging Data
Acquisition, Segmentation and Diagnosis for COVID-19 [71.41929762209328]
The pandemic of coronavirus disease 2019 (COVID-19) is spreading all over the world.
Medical imaging such as X-ray and computed tomography (CT) plays an essential role in the global fight against COVID-19.
The recently emerging artificial intelligence (AI) technologies further strengthen the power of the imaging tools and help medical specialists.
arXiv Detail & Related papers (2020-04-06T15:21:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.