AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation

• URL: http://arxiv.org/abs/2404.10714v1
• Date: Tue, 16 Apr 2024 16:43:36 GMT
• Title: AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation
• Authors: Zexin Li, Yiyang Lin, Zijie Fang, Shuyan Li, Xiu Li,
• Abstract summary: We develop an Attention-Based Key Region Selection Module, which can attend to regions with higher translation difficulty. We then develop a Varifocal Module to translate these regions at multiple resolutions. Experimental results show that our proposed AV-GAN outperforms existing image translation methods.
• Score: 25.665817446819386
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Different types of staining highlight different structures in organs, thereby assisting in diagnosis. However, due to the impossibility of repeated staining, we cannot obtain different types of stained slides of the same tissue area. Translating the slide that is easy to obtain (e.g., H&E) to slides of staining types difficult to obtain (e.g., MT, PAS) is a promising way to solve this problem. However, some regions are closely connected to other regions, and to maintain this connection, they often have complex structures and are difficult to translate, which may lead to wrong translations. In this paper, we propose the Attention-Based Varifocal Generative Adversarial Network (AV-GAN), which solves multiple problems in pathologic image translation tasks, such as uneven translation difficulty in different regions, mutual interference of multiple resolution information, and nuclear deformation. Specifically, we develop an Attention-Based Key Region Selection Module, which can attend to regions with higher translation difficulty. We then develop a Varifocal Module to translate these regions at multiple resolutions. Experimental results show that our proposed AV-GAN outperforms existing image translation methods with two virtual kidney tissue staining tasks and improves FID values by 15.9 and 4.16 respectively in the H&E-MT and H&E-PAS tasks.

Related papers

• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• M$^{2}$SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation [73.10707675345253]
  We propose a general multi-scale in multi-scale subtraction network (M$2$SNet) to finish diverse segmentation from medical image. Our method performs favorably against most state-of-the-art methods under different evaluation metrics on eleven datasets of four different medical image segmentation tasks.
  arXiv  Detail & Related papers  (2023-03-20T06:26:49Z)
• AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation [50.21065317817769]
  We propose an AlignTransformer framework, which includes the Align Hierarchical Attention (AHA) and the Multi-Grained Transformer (MGT) modules. Experiments on the public IU-Xray and MIMIC-CXR datasets show that the AlignTransformer can achieve results competitive with state-of-the-art methods on the two datasets.
  arXiv  Detail & Related papers  (2022-03-18T13:43:53Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning [4.32671477389424]
  Unsupervised domain adaptation based on image-to-image translation is gaining importance in digital pathology. We tackle the variation of different histological stains by unsupervised stain-to-stain translation. We use CycleGANs for stain-to-stain translation in kidney histopathology.
  arXiv  Detail & Related papers  (2021-12-16T12:42:40Z)
• TarGAN: Target-Aware Generative Adversarial Networks for Multi-modality Medical Image Translation [4.333115837538408]
  We propose a novel target-aware generative adversarial network called TarGAN. TarGAN is capable of learning multi-modality medical image translation without relying on paired data. Experiments on both quantitative measures and qualitative evaluations demonstrate that TarGAN outperforms the state-of-the-art methods in all cases.
  arXiv  Detail & Related papers  (2021-05-19T08:45:33Z)
• Multi-Texture GAN: Exploring the Multi-Scale Texture Translation for Brain MR Images [1.9163481966968943]
  A significant percentage of existing algorithms cannot explicitly exploit and preserve texture details from target scanners. In this paper, we design a multi-scale texture transfer to enrich the reconstruction images with more details. Our method achieves superior results in inter-protocol or inter-scanner translation over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-14T19:14:06Z)
• Towards Histopathological Stain Invariance by Unsupervised Domain Augmentation using Generative Adversarial Networks [0.7340845393655052]
  We propose an unsupervised augmentation approach based on adversarial image-to-image translation. By training the network on one commonly used staining modality and applying it to images that include corresponding, but differently stained, tissue structures, the presented method demonstrates significant improvements.
  arXiv  Detail & Related papers  (2020-12-22T23:32:17Z)
• Rethinking conditional GAN training: An approach using geometrically structured latent manifolds [58.07468272236356]
  Conditional GANs (cGAN) suffer from critical drawbacks such as the lack of diversity in generated outputs. We propose a novel training mechanism that increases both the diversity and the visual quality of a vanilla cGAN.
  arXiv  Detail & Related papers  (2020-11-25T22:54:11Z)
• Cross-Domain Medical Image Translation by Shared Latent Gaussian Mixture Model [10.05036157409819]
  Cross-domain image analysis tools are in high demand in real-world clinical applications. Current deep learning based segmentation models often generalize poorly between domains due to insufficient training data. We propose a patch-based model using shared latent variables from a Gaussian mixture model to preserve fine structures during medical image translation.
  arXiv  Detail & Related papers  (2020-07-14T17:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.