MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations

• URL: http://arxiv.org/abs/2411.10686v1
• Date: Sat, 16 Nov 2024 03:23:06 GMT
• Title: MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations
• Authors: Qixuan Jin, Walter Gerych, Marzyeh Ghassemi,
• Abstract summary: Masked Medical Image Inpainting (MaskMedPaint) We propose Masked Medical Image Inpainting (MaskMedPaint), which uses text-to-image diffusion models to augment training images by inpainting areas outside key classification regions to match the target domain. We demonstrate that MaskMedPaint enhances generalization to target domains across both natural (Waterbirds, iWildCam) and medical (ISIC 2018, Chest X-ray) datasets, given limited unlabeled target images.
• Score: 13.599251610827539
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spurious features associated with class labels can lead image classifiers to rely on shortcuts that don't generalize well to new domains. This is especially problematic in medical settings, where biased models fail when applied to different hospitals or systems. In such cases, data-driven methods to reduce spurious correlations are preferred, as clinicians can directly validate the modified images. While Denoising Diffusion Probabilistic Models (Diffusion Models) show promise for natural images, they are impractical for medical use due to the difficulty of describing spurious medical features. To address this, we propose Masked Medical Image Inpainting (MaskMedPaint), which uses text-to-image diffusion models to augment training images by inpainting areas outside key classification regions to match the target domain. We demonstrate that MaskMedPaint enhances generalization to target domains across both natural (Waterbirds, iWildCam) and medical (ISIC 2018, Chest X-ray) datasets, given limited unlabeled target images.

Related papers

• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation. We evaluate our method on three public longitudinal benchmark datasets of brain MRI and chest X-rays for counterfactual image generation.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• DAug: Diffusion-based Channel Augmentation for Radiology Image Retrieval and Classification [24.68697717585541]
  We propose a portable method that improves a perception model's performance with a generative model's output. Specifically, we extend a radiology image to multiple channels, with the additional channels being the heatmaps of regions where diseases tend to develop. Our method is motivated by the fact that generative models learn the distribution of normal and abnormal images, and such knowledge is complementary to image understanding tasks.
  arXiv  Detail & Related papers  (2024-12-06T07:43:28Z)
• COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images [3.5418498524791766]
  This research is development of a novel counterfactual inpainting approach (COIN) COIN flips the predicted classification label from abnormal to normal by using a generative model. The effectiveness of the method is demonstrated by segmenting synthetic targets and actual kidney tumors from CT images acquired from Tartu University Hospital in Estonia.
  arXiv  Detail & Related papers  (2024-04-19T12:09:49Z)
• Introducing Shape Prior Module in Diffusion Model for Medical Image Segmentation [7.7545714516743045]
  We propose an end-to-end framework called VerseDiff-UNet, which leverages the denoising diffusion probabilistic model (DDPM) Our approach integrates the diffusion model into a standard U-shaped architecture. We evaluate our method on a single dataset of spine images acquired through X-ray imaging.
  arXiv  Detail & Related papers  (2023-09-12T03:05:00Z)
• Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images [39.94162291765236]
  We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process.
  arXiv  Detail & Related papers  (2023-08-03T21:56:50Z)
• Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition [59.28732531600606]
  We propose a framework named textbfClass textbfAttention to textbfREgions of the lesion (CARE) to handle data imbalance issues. The CARE framework needs bounding boxes to represent the lesion regions of rare diseases. Results show that the CARE variants with automated bounding box generation are comparable to the original CARE framework.
  arXiv  Detail & Related papers  (2023-07-19T15:19:02Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classification [32.67098520984195]
  We propose the first diffusion-based model (named DiffMIC) to address general medical image classification. Our experimental results demonstrate that DiffMIC outperforms state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2023-03-19T09:15:45Z)
• MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer [53.575573940055335]
  We propose a novel Transformer-based Diffusion framework, called MedSegDiff-V2. We verify its effectiveness on 20 medical image segmentation tasks with different image modalities.
  arXiv  Detail & Related papers  (2023-01-19T03:42:36Z)
• Multitask Brain Tumor Inpainting with Diffusion Models: A Methodological Report [0.0]
  Inpainting algorithms are a subset of DL generative models that can alter one or more regions of an input image. The performance of these algorithms is frequently suboptimal due to their limited output variety. Denoising diffusion probabilistic models (DDPMs) are a recently introduced family of generative networks that can generate results of comparable quality to GANs.
  arXiv  Detail & Related papers  (2022-10-21T17:13:14Z)
• Cross-Modal Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop [63.81818077092879]
  We propose an end-to-end semi-supervised cross-modal contrastive learning framework for medical images. We first apply an image encoder to classify the chest X-rays and to generate the image features. The radiomic features are then passed through another dedicated encoder to act as the positive sample for the image features generated from the same chest X-ray.
  arXiv  Detail & Related papers  (2021-04-11T09:16:29Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.