BenchMD: A Benchmark for Unified Learning on Medical Images and Sensors

• URL: http://arxiv.org/abs/2304.08486v2
• Date: Mon, 26 Jun 2023 15:47:27 GMT
• Title: BenchMD: A Benchmark for Unified Learning on Medical Images and Sensors
• Authors: Kathryn Wantlin, Chenwei Wu, Shih-Cheng Huang, Oishi Banerjee, Farah Dadabhoy, Veeral Vipin Mehta, Ryan Wonhee Han, Fang Cao, Raja R. Narayan, Errol Colak, Adewole Adamson, Laura Heacock, Geoffrey H. Tison, Alex Tamkin, Pranav Rajpurkar
• Abstract summary: We present BenchMD, a benchmark that tests how well unified, modality-agnostic methods, including architectures and training techniques, perform on a diverse array of medical tasks. Our baseline results demonstrate that no unified learning technique achieves strong performance across all modalities, leaving ample room for improvement on the benchmark.
• Score: 8.695342954247606
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical data poses a daunting challenge for AI algorithms: it exists in many different modalities, experiences frequent distribution shifts, and suffers from a scarcity of examples and labels. Recent advances, including transformers and self-supervised learning, promise a more universal approach that can be applied flexibly across these diverse conditions. To measure and drive progress in this direction, we present BenchMD: a benchmark that tests how well unified, modality-agnostic methods, including architectures and training techniques (e.g. self-supervised learning, ImageNet pretraining),perform on a diverse array of clinically-relevant medical tasks. BenchMD combines 19 publicly available datasets for 7 medical modalities, including 1D sensor data, 2D images, and 3D volumetric scans. Our benchmark reflects real-world data constraints by evaluating methods across a range of dataset sizes, including challenging few-shot settings that incentivize the use of pretraining. Finally, we evaluate performance on out-of-distribution data collected at different hospitals than the training data, representing naturally-occurring distribution shifts that frequently degrade the performance of medical AI models. Our baseline results demonstrate that no unified learning technique achieves strong performance across all modalities, leaving ample room for improvement on the benchmark. Code is released at https://github.com/rajpurkarlab/BenchMD.

Related papers

• MIST: A Simple and Scalable End-To-End 3D Medical Imaging Segmentation Framework [1.4043931310479378]
  The Medical Imaging Toolkit (MIST) is designed to facilitate consistent training, testing, and evaluation of deep learning-based medical imaging segmentation methods. MIST standardizes data analysis, preprocessing, and evaluation pipelines, accommodating multiple architectures and loss functions.
  arXiv  Detail & Related papers  (2024-07-31T05:17:31Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning [36.33882718631217]
  Self-supervised learning is an efficient pre-training method for medical image analysis. We propose MedCoSS, a continuous self-supervised learning approach for multi-modal medical data. We conduct continuous self-supervised pre-training on a large-scale multi-modal unlabeled dataset.
  arXiv  Detail & Related papers  (2023-11-29T12:47:42Z)
• Tackling the Incomplete Annotation Issue in Universal Lesion Detection Task By Exploratory Training [10.627977735890191]
  Universal lesion detection has great value for clinical practice as it aims to detect lesions in multiple organs on medical images. Deep learning methods have shown promising results, but demanding large volumes of annotated data for training. We introduce a teacher-student detection model as basis, where the teacher's predictions are combined with incomplete annotations to train the student.
  arXiv  Detail & Related papers  (2023-09-23T08:44:07Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions [66.40971096248946]
  In this paper, we collect a series of MedISeg tricks for different model implementation phases. We experimentally explore the effectiveness of these tricks on consistent baselines. We also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play.
  arXiv  Detail & Related papers  (2022-09-21T12:30:05Z)
• Robust and Efficient Medical Imaging with Self-Supervision [80.62711706785834]
  We present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data.
  arXiv  Detail & Related papers  (2022-05-19T17:34:18Z)
• When Accuracy Meets Privacy: Two-Stage Federated Transfer Learning Framework in Classification of Medical Images on Limited Data: A COVID-19 Case Study [77.34726150561087]
  COVID-19 pandemic has spread rapidly and caused a shortage of global medical resources. CNN has been widely utilized and verified in analyzing medical images.
  arXiv  Detail & Related papers  (2022-03-24T02:09:41Z)
• CheXstray: Real-time Multi-Modal Data Concordance for Drift Detection in Medical Imaging AI [1.359138408203412]
  We build and test a medical imaging AI drift monitoring workflow that tracks data and model drift without contemporaneous ground truth. Key contributions include (1) proof-of-concept for medical imaging drift detection including use of VAE and domain specific statistical methods. This work has important implications for addressing the translation gap related to continuous medical imaging AI model monitoring in dynamic healthcare environments.
  arXiv  Detail & Related papers  (2022-02-06T18:58:35Z)
• Cross-Modal Information Maximization for Medical Imaging: CMIM [62.28852442561818]
  In hospitals, data are siloed to specific information systems that make the same information available under different modalities. This offers unique opportunities to obtain and use at train-time those multiple views of the same information that might not always be available at test-time. We propose an innovative framework that makes the most of available data by learning good representations of a multi-modal input that are resilient to modality dropping at test-time.
  arXiv  Detail & Related papers  (2020-10-20T20:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.