Reading Race: AI Recognises Patient's Racial Identity In Medical Images

• URL: http://arxiv.org/abs/2107.10356v1
• Date: Wed, 21 Jul 2021 21:10:16 GMT
• Title: Reading Race: AI Recognises Patient's Racial Identity In Medical Images
• Authors: Imon Banerjee, Ananth Reddy Bhimireddy, John L. Burns, Leo Anthony Celi, Li-Ching Chen, Ramon Correa, Natalie Dullerud, Marzyeh Ghassemi, Shih-Cheng Huang, Po-Chih Kuo, Matthew P Lungren, Lyle Palmer, Brandon J Price, Saptarshi Purkayastha, Ayis Pyrros, Luke Oakden-Rayner, Chima Okechukwu, Laleh Seyyed-Kalantari, Hari Trivedi, Ryan Wang, Zachary Zaiman, Haoran Zhang, Judy W Gichoya
• Abstract summary: There is no known correlation for race on medical imaging that would be obvious to the human expert interpreting the images. Standard deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities.
• Score: 9.287449389763413
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: In medical imaging, prior studies have demonstrated disparate AI performance by race, yet there is no known correlation for race on medical imaging that would be obvious to the human expert interpreting the images. Methods: Using private and public datasets we evaluate: A) performance quantification of deep learning models to detect race from medical images, including the ability of these models to generalize to external environments and across multiple imaging modalities, B) assessment of possible confounding anatomic and phenotype population features, such as disease distribution and body habitus as predictors of race, and C) investigation into the underlying mechanism by which AI models can recognize race. Findings: Standard deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities. Our findings hold under external validation conditions, as well as when models are optimized to perform clinically motivated tasks. We demonstrate this detection is not due to trivial proxies or imaging-related surrogate covariates for race, such as underlying disease distribution. Finally, we show that performance persists over all anatomical regions and frequency spectrum of the images suggesting that mitigation efforts will be challenging and demand further study. Interpretation: We emphasize that model ability to predict self-reported race is itself not the issue of importance. However, our findings that AI can trivially predict self-reported race -- even from corrupted, cropped, and noised medical images -- in a setting where clinical experts cannot, creates an enormous risk for all model deployments in medical imaging: if an AI model secretly used its knowledge of self-reported race to misclassify all Black patients, radiologists would not be able to tell using the same data the model has access to.

Related papers

• DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation [11.201840101870808]
  We propose an agent model capable of generating counterfactual images that prompt different decisions when plugged into a black box model. By employing this agent model, we can uncover influential image patterns that impact the black model's final predictions. We validated our approach in the rigorous domain of medical prognosis tasks.
  arXiv  Detail & Related papers  (2024-06-21T14:27:02Z)
• Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging [13.141767097232796]
  Self-supervised vision-language foundation models can detect a broad spectrum of pathologies without relying on explicit training annotations. It is crucial to ensure that these AI models do not mirror or amplify human biases, thereby disadvantaging historically marginalized groups such as females or Black patients. This study investigates the algorithmic fairness of state-of-the-art vision-language foundation models in chest X-ray diagnosis across five globally-sourced datasets.
  arXiv  Detail & Related papers  (2024-02-22T18:59:53Z)
• The Limits of Fair Medical Imaging AI In The Wild [43.97266228706059]
  We investigate the extent to which medical AI utilizes demographic encodings. We confirm that medical imaging AI leverages demographic shortcuts in disease classification. We find that models with less encoding of demographic attributes are often most "globally optimal"
  arXiv  Detail & Related papers  (2023-12-11T18:59:50Z)
• Deep Learning Discovery of Demographic Biomarkers in Echocardiography [0.3957768262206625]
  We test whether it is possible to predict age, race, and sex from cardiac ultrasound images using deep learning algorithms. We trained video-based convolutional neural networks to predict age, sex, and race. We found that deep learning models were able to identify age and sex, while unable to reliably predict race.
  arXiv  Detail & Related papers  (2022-07-13T16:48:49Z)
• RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR [14.586822005217485]
  We present RadFusion, a benchmark dataset of 1794 patients with corresponding EHR data and CT scans labeled for pulmonary embolism. Our results suggest that integrating imaging and EHR data can improve classification performance without introducing large disparities in the true positive rate between population groups.
  arXiv  Detail & Related papers  (2021-11-23T06:10:07Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
  A common scenario in developing countries is to have the volume metadata lost due multiple reasons. It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
  arXiv  Detail & Related papers  (2020-10-20T04:30:09Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.