A clinical validation of VinDr-CXR, an AI system for detecting abnormal chest radiographs

• URL: http://arxiv.org/abs/2104.02256v2
• Date: Wed, 7 Apr 2021 02:22:01 GMT
• Title: A clinical validation of VinDr-CXR, an AI system for detecting abnormal chest radiographs
• Authors: Ngoc Huy Nguyen, Ha Quy Nguyen, Nghia Trung Nguyen, Thang Viet Nguyen, Hieu Huy Pham, Tuan Ngoc-Minh Nguyen
• Abstract summary: We demonstrate a mechanism for validating an AI-based system for detecting abnormalities on X-ray scans. The system achieves an F1 score - the harmonic average of the recall and the precision - of 0.653 CI 0.635, 0.671) for detecting any abnormalities on chest X-rays.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computer-Aided Diagnosis (CAD) systems for chest radiographs using artificial intelligence (AI) have recently shown a great potential as a second opinion for radiologists. The performances of such systems, however, were mostly evaluated on a fixed dataset in a retrospective manner and, thus, far from the real performances in clinical practice. In this work, we demonstrate a mechanism for validating an AI-based system for detecting abnormalities on X-ray scans, VinDr-CXR, at the Phu Tho General Hospital - a provincial hospital in the North of Vietnam. The AI system was directly integrated into the Picture Archiving and Communication System (PACS) of the hospital after being trained on a fixed annotated dataset from other sources. The performance of the system was prospectively measured by matching and comparing the AI results with the radiology reports of 6,285 chest X-ray examinations extracted from the Hospital Information System (HIS) over the last two months of 2020. The normal/abnormal status of a radiology report was determined by a set of rules and served as the ground truth. Our system achieves an F1 score - the harmonic average of the recall and the precision - of 0.653 (95% CI 0.635, 0.671) for detecting any abnormalities on chest X-rays. Despite a significant drop from the in-lab performance, this result establishes a high level of confidence in applying such a system in real-life situations.

Related papers

• Large-scale Long-tailed Disease Diagnosis on Radiology Images [51.453990034460304]
  RadDiag is a foundational model supporting 2D and 3D inputs across various modalities and anatomies. Our dataset, RP3D-DiagDS, contains 40,936 cases with 195,010 scans covering 5,568 disorders.
  arXiv  Detail & Related papers  (2023-12-26T18:20:48Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Revisiting Computer-Aided Tuberculosis Diagnosis [56.80999479735375]
  Tuberculosis (TB) is a major global health threat, causing millions of deaths annually. Computer-aided tuberculosis diagnosis (CTD) using deep learning has shown promise, but progress is hindered by limited training data. We establish a large-scale dataset, namely the Tuberculosis X-ray (TBX11K) dataset, which contains 11,200 chest X-ray (CXR) images with corresponding bounding box annotations for TB areas. This dataset enables the training of sophisticated detectors for high-quality CTD.
  arXiv  Detail & Related papers  (2023-07-06T08:27:48Z)
• Artificial Intelligence System for Detection and Screening of Cardiac Abnormalities using Electrocardiogram Images [17.625471536027693]
  We present an AI system developed to detect and screen cardiac abnormalities (CAs) from real-world ECG images. The system was evaluated on a large dataset of 52,357 patients from multiple regions and populations across the world. Our study demonstrates the feasibility of an accurate, objective, easy-to-use, fast, and low-cost AI system for CA detection and screening.
  arXiv  Detail & Related papers  (2023-02-10T10:54:33Z)
• minoHealth.ai: A Clinical Evaluation Of Deep Learning Systems For the Diagnosis of Pleural Effusion and Cardiomegaly In Ghana, Vietnam and the United States of America [0.0]
  We evaluate how well minoHealth.ai systems, developed my minoHealth AI Labs, will perform at diagnosing cardiomegaly and pleural effusion. chest x-rays from Ghana, Vietnam and the USA, and how well AI systems will perform when compared with radiologists working in Ghana. For cardiomegaly, minoHealth.ai systems scored Area under the Receiver operating characteristic Curve (AUC-ROC) of 0.9 and 0.97 while the AUC-ROC of individual radiologists ranged from 0.77 to 0.86.
  arXiv  Detail & Related papers  (2022-10-31T20:12:41Z)
• An Accurate and Explainable Deep Learning System Improves Interobserver Agreement in the Interpretation of Chest Radiograph [0.33598755777055367]
  The VinDr-CXR can classify a CXR scan into multiple thoracic diseases and localize most types of critical findings on the image. The proposed system significantly improved the agreement between radiologists themselves with an increase of 1.5% in mean Fleiss' Kappa.
  arXiv  Detail & Related papers  (2022-08-06T17:03:49Z)
• StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder [48.2010192865749]
  Unsupervised anomaly detection (UAD) can learn a data distribution from an unlabelled dataset of healthy subjects and then be applied to detect out of distribution samples. This research proposes a compact version of the "context-encoding" VAE (ceVAE) model, combined with pre and post-processing steps, creating a UAD pipeline (StRegA) The proposed pipeline achieved a Dice score of 0.642$pm$0.101 while detecting tumours in T2w images of the BraTS dataset and 0.859$pm$0.112 while detecting artificially induced anomalies.
  arXiv  Detail & Related papers  (2022-01-31T14:27:35Z)
• Generative Residual Attention Network for Disease Detection [51.60842580044539]
  We present a novel approach for disease generation in X-rays using a conditional generative adversarial learning. We generate a corresponding radiology image in a target domain while preserving the identity of the patient. We then use the generated X-ray image in the target domain to augment our training to improve the detection performance.
  arXiv  Detail & Related papers  (2021-10-25T14:15:57Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Deep Learning for Distinguishing Normal versus Abnormal Chest Radiographs and Generalization to Unseen Diseases [7.93382570661604]
  We developed and evaluated an AI system to classify CXRs as normal or abnormal. Our results suggest that the AI system generalizes to new patient populations and abnormalities.
  arXiv  Detail & Related papers  (2020-10-22T01:52:51Z)
• Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning [17.315387269810426]
  We built a database containing more than 12,000 CXR scans and radiological reports. We developed a model based on deep convolutional neural network and recurrent network with attention mechanism. The model provides automated recognition of given scans and generation of reports.
  arXiv  Detail & Related papers  (2020-06-18T08:12:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.