Cross-Domain Image Synthesis: Generating H&E from Multiplex Biomarker Imaging

• URL: http://arxiv.org/abs/2508.04734v1
• Date: Tue, 05 Aug 2025 21:19:00 GMT
• Title: Cross-Domain Image Synthesis: Generating H&E from Multiplex Biomarker Imaging
• Authors: Jillur Rahman Saurav, Mohammad Sadegh Nasr, Jacob M. Luber,
• Abstract summary: We investigate the use of a multi-level Vector-Quantized Generative Adrial Network (VQGAN) to create high-fidelity virtual H&E stains from mIF images.<n>Our results show that while both architectures produce visually plausible stains, the virtual stains generated by our VQGAN provide a more effective substrate for computer-aided diagnosis.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: While multiplex immunofluorescence (mIF) imaging provides deep, spatially-resolved molecular data, integrating this information with the morphological standard of Hematoxylin & Eosin (H&E) can be very important for obtaining complementary information about the underlying tissue. Generating a virtual H&E stain from mIF data offers a powerful solution, providing immediate morphological context. Crucially, this approach enables the application of the vast ecosystem of H&E-based computer-aided diagnosis (CAD) tools to analyze rich molecular data, bridging the gap between molecular and morphological analysis. In this work, we investigate the use of a multi-level Vector-Quantized Generative Adversarial Network (VQGAN) to create high-fidelity virtual H&E stains from mIF images. We rigorously evaluated our VQGAN against a standard conditional GAN (cGAN) baseline on two publicly available colorectal cancer datasets, assessing performance on both image similarity and functional utility for downstream analysis. Our results show that while both architectures produce visually plausible images, the virtual stains generated by our VQGAN provide a more effective substrate for computer-aided diagnosis. Specifically, downstream nuclei segmentation and semantic preservation in tissue classification tasks performed on VQGAN-generated images demonstrate superior performance and agreement with ground-truth analysis compared to those from the cGAN. This work establishes that a multi-level VQGAN is a robust and superior architecture for generating scientifically useful virtual stains, offering a viable pathway to integrate the rich molecular data of mIF into established and powerful H&E-based analytical workflows.

Related papers

• Disentangled representations of microscopy images [0.9849635250118911]
  This work proposes a Disentangled Representation Learning (DRL) methodology to enhance model interpretability for microscopy image classification.<n>We show how a DRL framework, based on transferring a representation learnt from synthetic data, can provide a good trade-off between accuracy and interpretability in this domain.
  arXiv  Detail & Related papers  (2025-06-25T17:44:37Z)
• Virtual Staining of Label-Free Tissue in Imaging Mass Spectrometry [2.6165589247047465]
  We present a virtual histological staining approach that enhances spatial resolution and introduces cellular morphological contrast into mass spectrometry images. Blind testing on human kidney tissue demonstrated that the virtually stained images of label-free samples closely match their histochemically stained counterparts.
  arXiv  Detail & Related papers  (2024-11-20T08:30:11Z)
• Mew: Multiplexed Immunofluorescence Image Analysis through an Efficient Multiplex Network [84.88767228835928]
  We introduce Mew, a novel framework designed to efficiently process mIF images through the lens of multiplex network. Mew innovatively constructs a multiplex network comprising two distinct layers: a Voronoi network for geometric information and a Cell-type network for capturing cell-wise homogeneity. This framework equips a scalable and efficient Graph Neural Network (GNN), capable of processing the entire graph during training.
  arXiv  Detail & Related papers  (2024-07-25T08:22:30Z)
• ReStainGAN: Leveraging IHC to IF Stain Domain Translation for in-silico Data Generation [0.12564343689544843]
  In-silico datasets can expand the utility of existing annotations to new domains with different staining patterns in computational pathology. We propose a novel approach for the generation of in-silicochemistry (IHC) images by disentangling morphology specific IHC stains into separate image channels in immunofluorescence (IF) images.
  arXiv  Detail & Related papers  (2024-03-11T09:45:34Z)
• ADASR: An Adversarial Auto-Augmentation Framework for Hyperspectral and Multispectral Data Fusion [54.668445421149364]
  Deep learning-based hyperspectral image (HSI) super-resolution aims to generate high spatial resolution HSI (HR-HSI) by fusing hyperspectral image (HSI) and multispectral image (MSI) with deep neural networks (DNNs) In this letter, we propose a novel adversarial automatic data augmentation framework ADASR that automatically optimize and augments HSI-MSI sample pairs to enrich data diversity for HSI-MSI fusion.
  arXiv  Detail & Related papers  (2023-10-11T07:30:37Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation [48.638327652506284]
  Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. We present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach.
  arXiv  Detail & Related papers  (2022-11-12T05:39:17Z)
• A Self-attention Guided Multi-scale Gradient GAN for Diversified X-ray Image Synthesis [0.6308539010172307]
  Generative Adversarial Networks (GANs) are utilized to address the data limitation problem via the generation of synthetic images. Training challenges such as mode collapse, non-convergence, and instability degrade a GAN's performance in synthesizing diversified and high-quality images. This work proposes an attention-guided multi-scale gradient GAN architecture to model the relationship between long-range dependencies of biomedical image features.
  arXiv  Detail & Related papers  (2022-10-09T13:17:17Z)
• Preservation of High Frequency Content for Deep Learning-Based Medical Image Classification [74.84221280249876]
  An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information.
  arXiv  Detail & Related papers  (2022-05-08T15:29:54Z)
• MRI to PET Cross-Modality Translation using Globally and Locally Aware GAN (GLA-GAN) for Multi-Modal Diagnosis of Alzheimer's Disease [0.6597195879147557]
  generative adversarial networks (GANs) with the ability to synthesize realist images have shown great potential as an alternative to standard data augmentation techniques.<n>We propose a novel end-to-end, globally and locally aware image-to-image translation GAN (GLA-GAN) with a multi-path architecture that enforces both global structural integrity and fidelity to local details.
  arXiv  Detail & Related papers  (2021-08-04T16:38:33Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.