MoreStyle: Relax Low-frequency Constraint of Fourier-based Image Reconstruction in Generalizable Medical Image Segmentation

• URL: http://arxiv.org/abs/2403.11689v3
• Date: Mon, 1 Jul 2024 12:52:58 GMT
• Title: MoreStyle: Relax Low-frequency Constraint of Fourier-based Image Reconstruction in Generalizable Medical Image Segmentation
• Authors: Haoyu Zhao, Wenhui Dong, Rui Yu, Zhou Zhao, Du Bo, Yongchao Xu,
• Abstract summary: We introduce a Plug-and-Play module for data augmentation called MoreStyle. MoreStyle diversifies image styles by relaxing low-frequency constraints in Fourier space. With the help of adversarial learning, MoreStyle pinpoints the most intricate style combinations within latent features.
• Score: 53.24011398381715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of single-source domain generalization (SDG) in medical image segmentation is crucial due to frequent domain shifts in clinical image datasets. To address the challenge of poor generalization across different domains, we introduce a Plug-and-Play module for data augmentation called MoreStyle. MoreStyle diversifies image styles by relaxing low-frequency constraints in Fourier space, guiding the image reconstruction network. With the help of adversarial learning, MoreStyle further expands the style range and pinpoints the most intricate style combinations within latent features. To handle significant style variations, we introduce an uncertainty-weighted loss. This loss emphasizes hard-to-classify pixels resulting only from style shifts while mitigating true hard-to-classify pixels in both MoreStyle-generated and original images. Extensive experiments on two widely used benchmarks demonstrate that the proposed MoreStyle effectively helps to achieve good domain generalization ability, and has the potential to further boost the performance of some state-of-the-art SDG methods. Source code is available at https://github.com/zhaohaoyu376/morestyle.

Related papers

• Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization [21.591831983223997]
  We propose an exemplar-based style synthesis pipeline to improve domain generalization in semantic segmentation. Our method is based on a novel masked noise encoder for StyleGAN2 inversion. We achieve up to $12.4%$ mIoU improvements on driving-scene semantic segmentation under different types of data shifts.
  arXiv  Detail & Related papers  (2023-07-02T19:56:43Z)
• Intra-Source Style Augmentation for Improved Domain Generalization [21.591831983223997]
  We propose an intra-source style augmentation (ISSA) method to improve domain generalization in semantic segmentation. ISSA is model-agnostic and straightforwardly applicable with CNNs and Transformers. It is also complementary to other domain generalization techniques, e.g., it improves the recent state-of-the-art solution RobustNet by $3%$ mIoU in Cityscapes to Dark Z"urich.
  arXiv  Detail & Related papers  (2022-10-18T21:33:25Z)
• Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation [120.96012935286913]
  We propose a novel adversarial style augmentation approach, which can generate hard stylized images during training. Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains.
  arXiv  Detail & Related papers  (2022-07-11T14:01:25Z)
• MaxStyle: Adversarial Style Composition for Robust Medical Image Segmentation [12.329474646700776]
  Convolutional neural networks (CNNs) have achieved remarkable segmentation accuracy on benchmark datasets where training and test sets are from the same domain. CNNs' performance can degrade significantly on unseen domains, which hinders the deployment of CNNs in many clinical scenarios. We propose a novel data augmentation framework called MaxStyle, which maximizes the effectiveness of style augmentation for model OOD performance.
  arXiv  Detail & Related papers  (2022-06-02T21:37:21Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Style-Based Global Appearance Flow for Virtual Try-On [119.95115739956661]
  A novel global appearance flow estimation model is proposed in this work. Experiment results on a popular virtual try-on benchmark show that our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-03T10:58:04Z)
• Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning [95.78635058475439]
  Cross-domain few-shot learning aims at transferring knowledge from general nature images to novel domain-specific target categories. This paper studies the problem of CD-FSL by spanning the style distributions of the source dataset. To make our model robust to visual styles, the source images are augmented by swapping the styles of their low-frequency components with each other.
  arXiv  Detail & Related papers  (2022-03-15T05:36:41Z)
• Style-invariant Cardiac Image Segmentation with Test-time Augmentation [10.234493507401618]
  Deep models often suffer from severe performance drop due to the appearance shift in the real clinical setting. In this paper, we propose a novel style-invariant method for cardiac image segmentation.
  arXiv  Detail & Related papers  (2020-09-24T08:27:40Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.