A Concept-based Interpretable Model for the Diagnosis of Choroid Neoplasias using Multimodal Data

• URL: http://arxiv.org/abs/2403.05606v1
• Date: Fri, 8 Mar 2024 07:15:53 GMT
• Title: A Concept-based Interpretable Model for the Diagnosis of Choroid Neoplasias using Multimodal Data
• Authors: Yifan Wu, Yang Liu, Yue Yang, Michael S. Yao, Wenli Yang, Xuehui Shi, Lihong Yang, Dongjun Li, Yueming Liu, James C. Gee, Xuan Yang, Wenbin Wei, Shi Gu
• Abstract summary: We focus on choroid neoplasias, the most prevalent form of eye cancer in adults, albeit rare with 5.1 per million. Our work introduces a concept-based interpretable model that distinguishes between three types of choroidal tumors, integrating insights from domain experts via radiological reports. Remarkably, this model achieves an F1 score of 0.91, rivaling that of black-box models, but also boosts the diagnostic accuracy of junior doctors by 42%.
• Score: 28.632437578685842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diagnosing rare diseases presents a common challenge in clinical practice, necessitating the expertise of specialists for accurate identification. The advent of machine learning offers a promising solution, while the development of such technologies is hindered by the scarcity of data on rare conditions and the demand for models that are both interpretable and trustworthy in a clinical context. Interpretable AI, with its capacity for human-readable outputs, can facilitate validation by clinicians and contribute to medical education. In the current work, we focus on choroid neoplasias, the most prevalent form of eye cancer in adults, albeit rare with 5.1 per million. We built the so-far largest dataset consisting of 750 patients, incorporating three distinct imaging modalities collected from 2004 to 2022. Our work introduces a concept-based interpretable model that distinguishes between three types of choroidal tumors, integrating insights from domain experts via radiological reports. Remarkably, this model not only achieves an F1 score of 0.91, rivaling that of black-box models, but also boosts the diagnostic accuracy of junior doctors by 42%. This study highlights the significant potential of interpretable machine learning in improving the diagnosis of rare diseases, laying a groundwork for future breakthroughs in medical AI that could tackle a wider array of complex health scenarios.

Related papers

• Knowledge-driven AI-generated data for accurate and interpretable breast ultrasound diagnoses [29.70102468004044]
  We introduce a pipeline, TAILOR, that builds a knowledge-driven generative model to produce tailored synthetic data. The generative model, using 3,749 lesions as source data, can generate millions of breast-US images, especially for error-prone rare cases. In the prospective external evaluation, our diagnostic model outperforms the average performance of nine radiologists by 33.5% in specificity with the same sensitivity.
  arXiv  Detail & Related papers  (2024-07-23T16:49:01Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• Summarizing Radiology Reports Findings into Impressions [1.8964110318127383]
  We present a model with state-of-art radiology report summarization performance. We also provide an analysis of the model limitations and radiology knowledge gain. Our best performing model was a fine-tuned BERT-to-BERT encoder-decoder with 58.75/100 ROUGE-L F1.
  arXiv  Detail & Related papers  (2024-05-10T20:29:25Z)
• Advancing Multimodal Medical Capabilities of Gemini [32.28727204275662]
  We develop several models within the new Med-Gemini family that inherit core capabilities of Gemini. Med-Gemini-2D sets a new standard for AI-based chest X-ray (CXR) report generation based on expert evaluation. Med-Gemini-3D is the first ever large multimodal model-based report generation for 3D computed tomography (CT) volumes.
  arXiv  Detail & Related papers  (2024-05-06T04:44:22Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• Knowledge-Informed Machine Learning for Cancer Diagnosis and Prognosis: A review [2.2268038840298714]
  We review the state-of-the-art machine learning studies that adopted the fusion of biomedical knowledge and data. We provide an overview of diverse forms of knowledge representation and current strategies of knowledge integration into machine learning pipelines.
  arXiv  Detail & Related papers  (2024-01-12T07:01:36Z)
• Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining [12.656718786788758]
  We propose a self-conditioned diffusion model termed DiffChest and train it on a dataset of chest radiographs. DiffChest explains classifications on a patient-specific level and visualizes the confounding factors that may mislead the model. Our findings highlight the potential of pretraining based on diffusion models in medical image classification.
  arXiv  Detail & Related papers  (2023-09-29T10:38:08Z)
• BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks [68.39821375903591]
  Generalist AI holds the potential to address limitations due to its versatility in interpreting different data types. Here, we propose BiomedGPT, the first open-source and lightweight vision-language foundation model.
  arXiv  Detail & Related papers  (2023-05-26T17:14:43Z)
• Generative models improve fairness of medical classifiers under distribution shifts [49.10233060774818]
  We show that learning realistic augmentations automatically from data is possible in a label-efficient manner using generative models. We demonstrate that these learned augmentations can surpass ones by making models more robust and statistically fair in- and out-of-distribution.
  arXiv  Detail & Related papers  (2023-04-18T18:15:38Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.