MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications

• URL: http://arxiv.org/abs/2410.15432v1
• Date: Sun, 20 Oct 2024 16:03:55 GMT
• Title: MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications
• Authors: Yongrui Yu, Yannian Gu, Shaoting Zhang, Xiaofan Zhang,
• Abstract summary: This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM. MedDiff-FM leverages 3D CT images from multiple publicly available datasets, covering anatomical regions from head to abdomen, to pre-train a diffusion foundation model. Experimental results demonstrate the effectiveness of MedDiff-FM in addressing diverse downstream medical image tasks.
• Score: 10.321593505248341
• License:
• Abstract: Diffusion models have achieved significant success in both the natural image and medical image domains, encompassing a wide range of applications. Previous investigations in medical images have often been constrained to specific anatomical regions, particular applications, and limited datasets, resulting in isolated diffusion models. This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM. MedDiff-FM leverages 3D CT images from multiple publicly available datasets, covering anatomical regions from head to abdomen, to pre-train a diffusion foundation model, and explores the capabilities of the diffusion foundation model across a variety of application scenarios. The diffusion foundation model handles multi-level image processing both at the image-level and patch-level, and utilizes position embedding to establish multi-level spatial relationships as well as anatomical structures and region classes to control certain anatomical regions. MedDiff-FM manages several downstream tasks seamlessly, including image denoising, anomaly detection, and image synthesis. MedDiff-FM is also capable of performing lesion generation and lesion inpainting by rapidly fine-tuning the diffusion foundation model using ControlNet with task-specific conditions. Experimental results demonstrate the effectiveness of MedDiff-FM in addressing diverse downstream medical image tasks.

Related papers

• vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation [2.167228869154864]
  We present vesselFM, a foundation model designed specifically for the broad task of 3D blood vessel segmentation. Unlike previous models, vesselFM can effortlessly generalize to unseen domains.
  arXiv  Detail & Related papers  (2024-11-26T12:44:42Z)
• LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models [59.961172635689664]
  "Knowledge Decomposition" aims to improve the performance on specific medical tasks. We propose a novel framework named Low-Rank Knowledge Decomposition (LoRKD) LoRKD explicitly separates gradients from different tasks by incorporating low-rank expert modules and efficient knowledge separation convolution.
  arXiv  Detail & Related papers  (2024-09-29T03:56:21Z)
• Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation? [10.20366295974822]
  We introduce a novel decode head architecture, HQHSAM, which simply integrates elements from two state-of-the-art decoder heads, HSAM and HQSAM, to enhance segmentation performance. Our experiments on multiple datasets, encompassing various anatomies and modalities, reveal that FMs, particularly with the HQHSAM decode head, improve domain generalization for medical image segmentation.
  arXiv  Detail & Related papers  (2024-09-12T11:41:35Z)
• FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Models [54.09244105445476]
  This study introduces a novel knowledge injection approach, FedKIM, to scale the medical foundation model within a federated learning framework. FedKIM leverages lightweight local models to extract healthcare knowledge from private data and integrates this knowledge into a centralized foundation model. Our experiments across twelve tasks in seven modalities demonstrate the effectiveness of FedKIM in various settings.
  arXiv  Detail & Related papers  (2024-08-17T15:42:29Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• Towards Large-Scale Training of Pathology Foundation Models [1.5861468117231254]
  We release and make publicly available the first batch of our pathology FMs trained on open-access TCGA whole slide images. The experimental evaluation shows that our models reach state-of-the-art performance on various patch-level downstream tasks. We present an open-source framework designed for the consistent evaluation of pathology FMs across various downstream tasks.
  arXiv  Detail & Related papers  (2024-03-24T21:34:36Z)
• Introducing Shape Prior Module in Diffusion Model for Medical Image Segmentation [7.7545714516743045]
  We propose an end-to-end framework called VerseDiff-UNet, which leverages the denoising diffusion probabilistic model (DDPM) Our approach integrates the diffusion model into a standard U-shaped architecture. We evaluate our method on a single dataset of spine images acquired through X-ray imaging.
  arXiv  Detail & Related papers  (2023-09-12T03:05:00Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classification [32.67098520984195]
  We propose the first diffusion-based model (named DiffMIC) to address general medical image classification. Our experimental results demonstrate that DiffMIC outperforms state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2023-03-19T09:15:45Z)
• MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer [53.575573940055335]
  We propose a novel Transformer-based Diffusion framework, called MedSegDiff-V2. We verify its effectiveness on 20 medical image segmentation tasks with different image modalities.
  arXiv  Detail & Related papers  (2023-01-19T03:42:36Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.