VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stains

• URL: http://arxiv.org/abs/2407.19113v1
• Date: Fri, 26 Jul 2024 22:23:45 GMT
• Title: VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stains
• Authors: Shikha Dubey, Yosep Chong, Beatrice Knudsen, Shireen Y. Elhabian,
• Abstract summary: IHC stains are crucial in pathology practice for resolving complex diagnostic questions and guiding patient treatment decisions. Small biopsies often lack sufficient tissue for multiple stains while preserving material for subsequent molecular testing. VIMs is the first model to address this need, leveraging a large vision-language single-step diffusion model for virtual IHC multiplexing.
• Score: 0.9920087186610302
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a Virtual Immunohistochemistry Multiplex staining (VIMs) model designed to generate multiple immunohistochemistry (IHC) stains from a single hematoxylin and eosin (H&E) stained tissue section. IHC stains are crucial in pathology practice for resolving complex diagnostic questions and guiding patient treatment decisions. While commercial laboratories offer a wide array of up to 400 different antibody-based IHC stains, small biopsies often lack sufficient tissue for multiple stains while preserving material for subsequent molecular testing. This highlights the need for virtual IHC staining. Notably, VIMs is the first model to address this need, leveraging a large vision-language single-step diffusion model for virtual IHC multiplexing through text prompts for each IHC marker. VIMs is trained on uniplex paired H&E and IHC images, employing an adversarial training module. Testing of VIMs includes both paired and unpaired image sets. To enhance computational efficiency, VIMs utilizes a pre-trained large latent diffusion model fine-tuned with small, trainable weights through the Low-Rank Adapter (LoRA) approach. Experiments on nuclear and cytoplasmic IHC markers demonstrate that VIMs outperforms the base diffusion model and achieves performance comparable to Pix2Pix, a standard generative model for paired image translation. Multiple evaluation methods, including assessments by two pathologists, are used to determine the performance of VIMs. Additionally, experiments with different prompts highlight the impact of text conditioning. This paper represents the first attempt to accelerate histopathology research by demonstrating the generation of multiple IHC stains from a single H&E input using a single model trained solely on uniplex data.

Related papers

• IHC Matters: Incorporating IHC analysis to H&E Whole Slide Image Analysis for Improved Cancer Grading via Two-stage Multimodal Bilinear Pooling Fusion [19.813558168408047]
  We show that IHC and H&E possess distinct advantages and disadvantages while possessing certain complementary qualities. We develop a two-stage multi-modal bilinear model with a feature pooling module. Experiments demonstrate that incorporating IHC data into machine learning models, alongside H&E stained images, leads to superior predictive results for cancer grading.
  arXiv  Detail & Related papers  (2024-05-13T21:21:44Z)
• StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining [1.9029890402585894]
  Hematoxylin and Eosin (H&E) staining is the most commonly used for disease diagnosis and tumor recurrence tracking. Deep learning models have made Image-to-Image (I2I) translation a key research area, reducing the need for expensive physical staining processes. We propose StainDiffuser, a novel dual diffusion architecture for virtual staining that converges under a limited training budget.
  arXiv  Detail & Related papers  (2024-03-17T20:47:52Z)
• Automated segmentation of rheumatoid arthritis immunohistochemistry stained synovial tissue [0.0]
  Rheumatoid Arthritis (RA) is a chronic, autoimmune disease which primarily affects the joint's synovial tissue. It is a highly heterogeneous disease, with wide cellular and molecular variability observed in synovial tissues. We train a UNET on a hand-curated, real-world multi-centre clinical dataset R4RA, which contains multiple types of IHC staining. The model obtains a DICE score of 0.865 and successfully segments different types of IHC staining, as well as dealing with variance in colours, intensity and common WSIs artefacts from the different clinical centres.
  arXiv  Detail & Related papers  (2023-09-13T18:43:14Z)
• Structural Cycle GAN for Virtual Immunohistochemistry Staining of Gland Markers in the Colon [1.741980945827445]
  Hematoxylin and Eosin (H&E) staining is one of the most frequently used stains for disease analysis, diagnosis, and grading. Pathologists do need differentchemical (IHC) stains to analyze specific structures or cells. Hematoxylin and Eosin (H&E) staining is one of the most frequently used stains for disease analysis, diagnosis, and grading.
  arXiv  Detail & Related papers  (2023-08-25T05:24:23Z)
• Adaptive Supervised PatchNCE Loss for Learning H&E-to-IHC Stain Translation with Inconsistent Groundtruth Image Pairs [5.841841666625825]
  We present a new loss function, Adaptive Supervised PatchNCE (ASP), to deal with the input to target inconsistencies in a proposed H&E-to-IHC image-to-image translation framework. In our experiment, we demonstrate that our proposed method outperforms existing image-to-image translation methods for stain translation to multiple IHC stains.
  arXiv  Detail & Related papers  (2023-03-10T19:56:34Z)
• GraVIS: Grouping Augmented Views from Independent Sources for Dermatology Analysis [52.04899592688968]
  We propose GraVIS, which is specifically optimized for learning self-supervised features from dermatology images. GraVIS significantly outperforms its transfer learning and self-supervised learning counterparts in both lesion segmentation and disease classification tasks.
  arXiv  Detail & Related papers  (2023-01-11T11:38:37Z)
• Stain-invariant self supervised learning for histopathology image analysis [74.98663573628743]
  We present a self-supervised algorithm for several classification tasks within hematoxylin and eosin stained images of breast cancer. Our method achieves the state-of-the-art performance on several publicly available breast cancer datasets.
  arXiv  Detail & Related papers  (2022-11-14T18:16:36Z)
• Lymphocyte Classification in Hyperspectral Images of Ovarian Cancer Tissue Biopsy Samples [94.37521840642141]
  We present a machine learning pipeline to segment white blood cell pixels in hyperspectral images of biopsy cores. These cells are clinically important for diagnosis, but some prior work has struggled to incorporate them due to difficulty obtaining precise pixel labels.
  arXiv  Detail & Related papers  (2022-03-23T00:58:27Z)
• M3Lung-Sys: A Deep Learning System for Multi-Class Lung Pneumonia Screening from CT Imaging [85.00066186644466]
  We propose a Multi-task Multi-slice Deep Learning System (M3Lung-Sys) for multi-class lung pneumonia screening from CT imaging. In addition to distinguish COVID-19 from Healthy, H1N1, and CAP cases, our M 3 Lung-Sys also be able to locate the areas of relevant lesions.
  arXiv  Detail & Related papers  (2020-10-07T06:22:24Z)
• Diagnosis of Coronavirus Disease 2019 (COVID-19) with Structured Latent Multi-View Representation Learning [48.05232274463484]
  Recently, the outbreak of Coronavirus Disease 2019 (COVID-19) has spread rapidly across the world. Due to the large number of affected patients and heavy labor for doctors, computer-aided diagnosis with machine learning algorithm is urgently needed. In this study, we propose to conduct the diagnosis of COVID-19 with a series of features extracted from CT images.
  arXiv  Detail & Related papers  (2020-05-06T15:19:15Z)
• Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification [76.89093364969253]
  We propose to apply Few-Shot Learning to skin disease identification to address the extreme scarcity of training sample problem. Based on a detailed analysis, we propose the Query-Relative (QR) loss, which proves superior to Cross Entropy (CE) under episode training. We further strengthen the proposed QR loss with a novel adaptive hard margin strategy.
  arXiv  Detail & Related papers  (2020-04-21T00:57:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.