Multi stain graph fusion for multimodal integration in pathology

• URL: http://arxiv.org/abs/2204.12541v1
• Date: Tue, 26 Apr 2022 18:49:24 GMT
• Title: Multi stain graph fusion for multimodal integration in pathology
• Authors: Chaitanya Dwivedi, Shima Nofallah, Maryam Pouryahya, Janani Iyer, Kenneth Leidal, Chuhan Chung, Timothy Watkins, Andrew Billin, Robert Myers, John Abel, Ali Behrooz
• Abstract summary: We introduce a multimodal CNN-GNN based graph fusion approach to predict pathologic scores. We demonstrate this approach in nonalcoholic steatohepatitis (NASH) by predicting CRN fibrosis stage and NAFLD Activity Score (NAS) We report up to 20% improvement in predicting fibrosis stage and NAS component grades over single-stain modeling approaches.
• Score: 0.5942143381145052
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In pathology, tissue samples are assessed using multiple staining techniques to enhance contrast in unique histologic features. In this paper, we introduce a multimodal CNN-GNN based graph fusion approach that leverages complementary information from multiple non-registered histopathology images to predict pathologic scores. We demonstrate this approach in nonalcoholic steatohepatitis (NASH) by predicting CRN fibrosis stage and NAFLD Activity Score (NAS). Primary assessment of NASH typically requires liver biopsy evaluation on two histological stains: Trichrome (TC) and hematoxylin and eosin (H&E). Our multimodal approach learns to extract complementary information from TC and H&E graphs corresponding to each stain while simultaneously learning an optimal policy to combine this information. We report up to 20% improvement in predicting fibrosis stage and NAS component grades over single-stain modeling approaches, measured by computing linearly weighted Cohen's kappa between machine-derived vs. pathologist consensus scores. Broadly, this paper demonstrates the value of leveraging diverse pathology images for improved ML-powered histologic assessment.

Related papers

• Graph-Ensemble Learning Model for Multi-label Skin Lesion Classification using Dermoscopy and Clinical Images [7.159532626507458]
  This study introduces a Graph Convolution Network (GCN) to exploit prior co-occurrence between each category as a correlation matrix into the deep learning model for the multi-label classification. We propose a Graph-Ensemble Learning Model (GELN) that views the prediction from GCN as complementary information of the predictions from the fusion model.
  arXiv  Detail & Related papers  (2023-07-04T13:19:57Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Semi-supervised GAN for Bladder Tissue Classification in Multi-Domain Endoscopic Images [10.48945682277992]
  We propose a semi-surprised Generative Adrial Network (GAN)-based method composed of three main components. The overall average classification accuracy, precision, and recall obtained with the proposed method for tissue classification are 0.90, 0.88, and 0.89 respectively.
  arXiv  Detail & Related papers  (2022-12-21T21:32:36Z)
• Stain based contrastive co-training for histopathological image analysis [61.87751502143719]
  We propose a novel semi-supervised learning approach for classification of histovolution images. We employ strong supervision with patch-level annotations combined with a novel co-training loss to create a semi-supervised learning framework. We evaluate our approach in clear cell renal cell and prostate carcinomas, and demonstrate improvement over state-of-the-art semi-supervised learning methods.
  arXiv  Detail & Related papers  (2022-06-24T22:25:31Z)
• Lymphocyte Classification in Hyperspectral Images of Ovarian Cancer Tissue Biopsy Samples [94.37521840642141]
  We present a machine learning pipeline to segment white blood cell pixels in hyperspectral images of biopsy cores. These cells are clinically important for diagnosis, but some prior work has struggled to incorporate them due to difficulty obtaining precise pixel labels.
  arXiv  Detail & Related papers  (2022-03-23T00:58:27Z)
• Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis [88.39466012709205]
  This paper builds a novel medical slice to increase the between-slice resolution. Considering that the ground-truth intermediate medical slices are always absent in clinical practice, we introduce the incremental cross-view mutual distillation strategy. Our method outperforms state-of-the-art algorithms by clear margins.
  arXiv  Detail & Related papers  (2021-12-20T03:38:37Z)
• Lung Cancer Lesion Detection in Histopathology Images Using Graph-Based Sparse PCA Network [93.22587316229954]
  We propose a graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E) We evaluate the performance of the proposed algorithm on H&E slides obtained from an SVM K-rasG12D lung cancer mouse model using precision/recall rates, F-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC)
  arXiv  Detail & Related papers  (2021-10-27T19:28:36Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Multi-Contrast Computed Tomography Healthy Kidney Atlas [3.0066242826634415]
  A volumetric atlas framework is needed to integrate and visualize the variability across scales. There is no abdominal and retroperitoneal organs atlas framework for multi-contrast CT. We introduce a deep learning-based volume of interest extraction method and an automated two-stage hierarchal registration pipeline to register abdominal volumes to a high-resolution CT atlas template.
  arXiv  Detail & Related papers  (2020-12-23T01:03:16Z)
• StyPath: Style-Transfer Data Augmentation For Robust Histology Image Classification [6.690876060631452]
  We propose a novel pipeline to build robust deep neural networks for AMR classification based on StyPath. Each image was generated in 1.84 + 0.03 seconds using a single GTX V TITAN and pytorch. Our results imply that our style-transfer augmentation technique improves histological classification performance.
  arXiv  Detail & Related papers  (2020-07-09T18:02:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.