MedPix 2.0: A Comprehensive Multimodal Biomedical Dataset for Advanced AI Applications

• URL: http://arxiv.org/abs/2407.02994v1
• Date: Wed, 3 Jul 2024 10:49:21 GMT
• Title: MedPix 2.0: A Comprehensive Multimodal Biomedical Dataset for Advanced AI Applications
• Authors: Irene Siragusa, Salvatore Contino, Massimo La Ciura, Rosario Alicata, Roberto Pirrone,
• Abstract summary: This paper illustrates the entire workflow for building the data set MedPix 2.0. Along with the dataset, we developed a GUI aimed at navigating efficiently the MongoDB instance. We also propose a CLIP-based model trained on MedPix 2.0 for scan classification tasks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing interest in developing Artificial Intelligence applications in the medical domain, suffers from the lack of high-quality dataset, mainly due to privacy-related issues. Moreover, the recent rising of Multimodal Large Language Models (MLLM) leads to a need for multimodal medical datasets, where clinical reports and findings are attached to the corresponding CT or MR scans. This paper illustrates the entire workflow for building the data set MedPix 2.0. Starting from the well-known multimodal dataset MedPix\textsuperscript{\textregistered}, mainly used by physicians, nurses and healthcare students for Continuing Medical Education purposes, a semi-automatic pipeline was developed to extract visual and textual data followed by a manual curing procedure where noisy samples were removed, thus creating a MongoDB database. Along with the dataset, we developed a GUI aimed at navigating efficiently the MongoDB instance, and obtaining the raw data that can be easily used for training and/or fine-tuning MLLMs. To enforce this point, we also propose a CLIP-based model trained on MedPix 2.0 for scan classification tasks.

Related papers

• MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression [9.739199023618042]
  We introduce MMXU, a novel dataset for MedVQA that focuses on identifying changes in specific regions between two patient visits. Unlike previous datasets that primarily address single-image questions, MMXU enables multi-image questions, incorporating both current and historical patient data. Our experiments show that integrating historical records significantly enhances diagnostic accuracy by at least 20%, bridging the gap between current LVLMs and human expert performance.
  arXiv  Detail & Related papers  (2025-02-17T10:43:38Z)
• UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities [68.12889379702824]
  Vision-Language Models (VLMs) trained via contrastive learning have achieved notable success in natural image tasks. UniMed is a large-scale, open-source multi-modal medical dataset comprising over 5.3 million image-text pairs. We trained UniMed-CLIP, a unified VLM for six modalities, achieving notable gains in zero-shot evaluations.
  arXiv  Detail & Related papers  (2024-12-13T18:59:40Z)
• Medical Vision-Language Pre-Training for Brain Abnormalities [96.1408455065347]
  We show how to automatically collect medical image-text aligned data for pretraining from public resources such as PubMed. In particular, we present a pipeline that streamlines the pre-training process by initially collecting a large brain image-text dataset. We also investigate the unique challenge of mapping subfigures to subcaptions in the medical domain.
  arXiv  Detail & Related papers  (2024-04-27T05:03:42Z)
• All-in-one platform for AI R&D in medical imaging, encompassing data collection, selection, annotation, and pre-processing [0.6291643559814802]
  Deep Learning is advancing medical imaging Research and Development (R&D), leading to the frequent clinical use of Artificial Intelligence/Machine Learning (AI/ML)-based medical devices. However, to advance AI R&D, two challenges arise: 1) significant data imbalance, with most data from Europe/America and under 10% from Asia, despite its 60% global population share; and 2) hefty time and investment needed to curate datasets for commercial use. In response, we established the first commercial medical imaging platform, encompassing steps like: 1) data collection, 2) data selection, 3) annotation, and 4) pre-processing.
  arXiv  Detail & Related papers  (2024-03-10T09:24:53Z)
• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data [66.9359934608229]
  This study aims to initiate the development of Radiology Foundation Model, termed as RadFM. To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans. We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
  arXiv  Detail & Related papers  (2023-08-04T17:00:38Z)
• Med-Flamingo: a Multimodal Medical Few-shot Learner [58.85676013818811]
  We propose Med-Flamingo, a multimodal few-shot learner adapted to the medical domain. Based on OpenFlamingo-9B, we continue pre-training on paired and interleaved medical image-text data from publications and textbooks. We conduct the first human evaluation for generative medical VQA where physicians review the problems and blinded generations in an interactive app.
  arXiv  Detail & Related papers  (2023-07-27T20:36:02Z)
• medigan: A Python Library of Pretrained Generative Models for Enriched Data Access in Medical Imaging [3.8568465270960264]
  medigan is a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library. It allows researchers and developers to create, increase, and domain-adapt their training data in just a few lines of code. The library's scalability and design is demonstrated by its growing number of integrated and readily-usable pretrained generative models.
  arXiv  Detail & Related papers  (2022-09-28T23:45:33Z)
• Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions [66.40971096248946]
  In this paper, we collect a series of MedISeg tricks for different model implementation phases. We experimentally explore the effectiveness of these tricks on consistent baselines. We also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play.
  arXiv  Detail & Related papers  (2022-09-21T12:30:05Z)
• MedMNIST v2: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification [59.10015984688104]
  MedMNIST v2 is a large-scale MNIST-like dataset collection of standardized biomedical images. The resulting dataset consists of 708,069 2D images and 10,214 3D images in total.
  arXiv  Detail & Related papers  (2021-10-27T22:02:04Z)
• MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis [46.02653153307692]
  We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28x28 images. MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis.
  arXiv  Detail & Related papers  (2020-10-28T12:41:18Z)
• ivadomed: A Medical Imaging Deep Learning Toolbox [3.6064670806006647]
  ivadomed is an open-source Python package for designing, end-to-end training, and evaluating deep learning models. The package includes APIs, command-line tools, documentation, and tutorials.
  arXiv  Detail & Related papers  (2020-10-20T03:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.