Multi-modal learning for predicting the genotype of glioma

• URL: http://arxiv.org/abs/2203.10852v1
• Date: Mon, 21 Mar 2022 10:20:04 GMT
• Title: Multi-modal learning for predicting the genotype of glioma
• Authors: Yiran Wei, Xi Chen, Lei Zhu, Lipei Zhang, Carola-Bibiane Sch\"onlieb, Stephen J. Price, Chao Li
• Abstract summary: The isocitrate dehydrogenase (IDH) gene mutation is an essential biomarker for the diagnosis and prognosis of glioma. It is promising to better predict glioma genotype by integrating focal tumor image and geometric features with brain network features derived from MRI. We propose a multi-modal learning framework using three separate encoders to extract features of focal tumor image, tumor geometrics and global brain networks.
• Score: 14.93152817415408
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The isocitrate dehydrogenase (IDH) gene mutation is an essential biomarker for the diagnosis and prognosis of glioma. It is promising to better predict glioma genotype by integrating focal tumor image and geometric features with brain network features derived from MRI. Convolutions neural networks show reasonable performance in predicting IDH mutation, which, however, cannot learn from non-Euclidean data, e.g., geometric and network data. In this study, we propose a multi-modal learning framework using three separate encoders to extract features of focal tumor image, tumor geometrics and global brain networks. To mitigate the limited availability of diffusion MRI, we develop a self-supervised approach to generate brain networks from anatomical multi-sequence MRI. Moreover, to extract tumor-related features from the brain network, we design a hierarchical attention module for the brain network encoder. Further, we design a bi-level multi-modal contrastive loss to align the multi-modal features and tackle the domain gap at the focal tumor and global brain. Finally, we propose a weighted population graph to integrate the multi-modal features for genotype prediction. Experimental results on the testing set show that the proposed model outperforms the baseline deep learning models. The ablation experiments validate the performance of different components of the framework. The visualized interpretation corresponds to clinical knowledge with further validation. In conclusion, the proposed learning framework provides a novel approach for predicting the genotype of glioma.

Related papers

• Enhancing Brain Tumor Classification Using TrAdaBoost and Multi-Classifier Deep Learning Approaches [0.0]
  Brain tumors pose a serious health threat due to their rapid growth and potential for metastasis. This study aims to improve the efficiency and accuracy of brain tumor classification. Our approach combines state-of-the-art deep learning algorithms, including the Vision Transformer (ViT), Capsule Neural Network (CapsNet), and convolutional neural networks (CNNs) such as ResNet-152 and VGG16.
  arXiv  Detail & Related papers  (2024-10-31T07:28:06Z)
• Pathology-genomic fusion via biologically informed cross-modality graph learning for survival analysis [7.996257103473235]
  We propose Pathology-Genome Heterogeneous Graph (PGHG) that integrates whole slide images (WSI) and bulk RNA-Seq expression data with heterogeneous graph neural network for cancer survival analysis. The PGHG consists of biological knowledge-guided representation learning network and pathology-genome heterogeneous graph. We evaluate the model on low-grade gliomas, glioblastoma, and kidney renal papillary cell carcinoma datasets from the Cancer Genome Atlas.
  arXiv  Detail & Related papers  (2024-04-11T09:07:40Z)
• Prediction of brain tumor recurrence location based on multi-modal fusion and nonlinear correlation learning [55.789874096142285]
  We present a deep learning-based brain tumor recurrence location prediction network. We first train a multi-modal brain tumor segmentation network on the public dataset BraTS 2021. Then, the pre-trained encoder is transferred to our private dataset for extracting the rich semantic features. Two decoders are constructed to jointly segment the present brain tumor and predict its future tumor recurrence location.
  arXiv  Detail & Related papers  (2023-04-11T02:45:38Z)
• Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI [55.78588835407174]
  We propose a method that reformulates the generation task of diffusion models as a patch-based estimation of healthy brain anatomy. We evaluate our approach on data of tumors and multiple sclerosis lesions and demonstrate a relative improvement of 25.1% compared to existing baselines.
  arXiv  Detail & Related papers  (2023-03-07T09:40:22Z)
• Collaborative learning of images and geometrics for predicting isocitrate dehydrogenase status of glioma [8.262398325144774]
  Gold standard of IDH mutation detection requires tumour tissue obtained via invasive approaches and is usually expensive. Recent advancement in radiogenomics provides a non-invasive approach for predicting IDH mutation based on MRI. Here we propose a collaborative learning framework that learns both tumor images and tumor geometrics using convolutional neural networks (CNN) and graph neural networks (GNN) Our results show that the proposed model outperforms the baseline model of 3D-DenseNet121.
  arXiv  Detail & Related papers  (2022-01-14T15:58:07Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)
• Predicting isocitrate dehydrogenase mutationstatus in glioma using structural brain networksand graph neural networks [6.67232502899311]
  The isocitrate dehydrogenase (IDH) gene mutation status provides critical diagnostic and prognostic value for glioma. Machine learning and deep learning models show reasonable performance in predicting IDH mutation status. We propose a method to predict the IDH mutation using graph neural networks (GNN) based on the structural brain network of patients.
  arXiv  Detail & Related papers  (2021-09-04T12:19:33Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction [33.03449099154264]
  Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. We propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating the fMRI time series and the functional connectivity (FC) between each pair of brain regions. We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score.
  arXiv  Detail & Related papers  (2021-01-20T20:53:07Z)
• M2Net: Multi-modal Multi-channel Network for Overall Survival Time Prediction of Brain Tumor Patients [151.4352001822956]
  Early and accurate prediction of overall survival (OS) time can help to obtain better treatment planning for brain tumor patients. Existing prediction methods rely on radiomic features at the local lesion area of a magnetic resonance (MR) volume. We propose an end-to-end OS time prediction model; namely, Multi-modal Multi-channel Network (M2Net)
  arXiv  Detail & Related papers  (2020-06-01T05:21:37Z)
• A Graph Gaussian Embedding Method for Predicting Alzheimer's Disease Progression with MEG Brain Networks [59.15734147867412]
  Characterizing the subtle changes of functional brain networks associated with Alzheimer's disease (AD) is important for early diagnosis and prediction of disease progression. We developed a new deep learning method, termed multiple graph Gaussian embedding model (MG2G) We used MG2G to detect the intrinsic latent dimensionality of MEG brain networks, predict the progression of patients with mild cognitive impairment (MCI) to AD, and identify brain regions with network alterations related to MCI.
  arXiv  Detail & Related papers  (2020-05-08T02:29:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.