medigan: A Python Library of Pretrained Generative Models for Enriched
Data Access in Medical Imaging
- URL: http://arxiv.org/abs/2209.14472v1
- Date: Wed, 28 Sep 2022 23:45:33 GMT
- Title: medigan: A Python Library of Pretrained Generative Models for Enriched
Data Access in Medical Imaging
- Authors: Richard Osuala, Grzegorz Skorupko, Noussair Lazrak, Lidia Garrucho,
Eloy Garc\'ia, Smriti Joshi, Socayna Jouide, Michael Rutherford, Fred Prior,
Kaisar Kushibar, Oliver Diaz, Karim Lekadir
- Abstract summary: medigan is a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library.
It allows researchers and developers to create, increase, and domain-adapt their training data in just a few lines of code.
The library's scalability and design is demonstrated by its growing number of integrated and readily-usable pretrained generative models.
- Score: 3.8568465270960264
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Synthetic data generated by generative models can enhance the performance and
capabilities of data-hungry deep learning models in medical imaging. However,
there is (1) limited availability of (synthetic) datasets and (2) generative
models are complex to train, which hinders their adoption in research and
clinical applications. To reduce this entry barrier, we propose medigan, a
one-stop shop for pretrained generative models implemented as an open-source
framework-agnostic Python library. medigan allows researchers and developers to
create, increase, and domain-adapt their training data in just a few lines of
code. Guided by design decisions based on gathered end-user requirements, we
implement medigan based on modular components for generative model (i)
execution, (ii) visualisation, (iii) search & ranking, and (iv) contribution.
The library's scalability and design is demonstrated by its growing number of
integrated and readily-usable pretrained generative models consisting of 21
models utilising 9 different Generative Adversarial Network architectures
trained on 11 datasets from 4 domains, namely, mammography, endoscopy, x-ray,
and MRI. Furthermore, 3 applications of medigan are analysed in this work,
which include (a) enabling community-wide sharing of restricted data, (b)
investigating generative model evaluation metrics, and (c) improving clinical
downstream tasks. In (b), extending on common medical image synthesis
assessment and reporting standards, we show Fr\'echet Inception Distance
variability based on image normalisation and radiology-specific feature
extraction.
Related papers
- Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges [2.1835659964186087]
This paper presents a systematic review of generative models used to synthesize various medical data types.
Our study encompasses a broad array of medical data modalities and explores various generative models.
arXiv Detail & Related papers (2024-06-27T14:00:11Z) - Medical Vision-Language Pre-Training for Brain Abnormalities [96.1408455065347]
We show how to automatically collect medical image-text aligned data for pretraining from public resources such as PubMed.
In particular, we present a pipeline that streamlines the pre-training process by initially collecting a large brain image-text dataset.
We also investigate the unique challenge of mapping subfigures to subcaptions in the medical domain.
arXiv Detail & Related papers (2024-04-27T05:03:42Z) - Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology.
For training, we assemble a large dataset of over 697 thousand radiology image-text pairs.
For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation.
The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
arXiv Detail & Related papers (2024-03-12T18:12:02Z) - Three-dimensional Bone Image Synthesis with Generative Adversarial
Networks [2.499907423888049]
This work demonstrates that three-dimensional generative adversarial networks (GANs) can be efficiently trained to generate high-resolution medical volumes.
GAN inversion is successfully implemented for the three-dimensional setting and used for extensive research on model interpretability.
arXiv Detail & Related papers (2023-10-26T08:08:17Z) - Towards Generalist Foundation Model for Radiology by Leveraging
Web-scale 2D&3D Medical Data [66.9359934608229]
This study aims to initiate the development of Radiology Foundation Model, termed as RadFM.
To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans.
We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
arXiv Detail & Related papers (2023-08-04T17:00:38Z) - Generative AI for Medical Imaging: extending the MONAI Framework [29.152928147704507]
We present MONAI Generative Models, a freely available open-source platform that allows researchers and developers to easily train, evaluate, and deploy generative models.
Our platform reproduces state-of-art studies in a standardised way involving different architectures.
We have implemented these models in a generalisable fashion, illustrating that their results can be extended to 2D or 3D scenarios.
arXiv Detail & Related papers (2023-07-27T21:58:26Z) - DeepMediX: A Deep Learning-Driven Resource-Efficient Medical Diagnosis
Across the Spectrum [15.382184404673389]
This work presents textttDeepMediX, a groundbreaking, resource-efficient model that significantly addresses this challenge.
Built on top of the MobileNetV2 architecture, DeepMediX excels in classifying brain MRI scans and skin cancer images.
DeepMediX's design also includes the concept of Federated Learning, enabling a collaborative learning approach without compromising data privacy.
arXiv Detail & Related papers (2023-07-01T12:30:58Z) - LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical
Imaging via Second-order Graph Matching [59.01894976615714]
We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets.
We have collected approximately 1.3 million medical images from 55 publicly available datasets.
LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
arXiv Detail & Related papers (2023-06-20T22:21:34Z) - Competence-based Multimodal Curriculum Learning for Medical Report
Generation [98.10763792453925]
We propose a Competence-based Multimodal Curriculum Learning framework ( CMCL) to alleviate the data bias and make best use of available data.
Specifically, CMCL simulates the learning process of radiologists and optimize the model in a step by step manner.
Experiments on the public IU-Xray and MIMIC-CXR datasets show that CMCL can be incorporated into existing models to improve their performance.
arXiv Detail & Related papers (2022-06-24T08:16:01Z) - Ensemble Transfer Learning for the Prediction of Anti-Cancer Drug
Response [49.86828302591469]
In this paper, we apply transfer learning to the prediction of anti-cancer drug response.
We apply the classic transfer learning framework that trains a prediction model on the source dataset and refines it on the target dataset.
The ensemble transfer learning pipeline is implemented using LightGBM and two deep neural network (DNN) models with different architectures.
arXiv Detail & Related papers (2020-05-13T20:29:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.