Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review

• URL: http://arxiv.org/abs/2303.06471v3
• Date: Thu, 28 Mar 2024 21:36:56 GMT
• Title: Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review
• Authors: Asim Waqas, Aakash Tripathi, Ravi P. Ramachandran, Paul Stewart, Ghulam Rasool,
• Abstract summary: Integrating diverse data types can improve the accuracy and reliability of cancer diagnosis and treatment. Deep neural networks have facilitated the development of sophisticated multimodal data fusion approaches. Recent deep learning frameworks such as Graph Neural Networks (GNNs) and Transformers have shown remarkable success in multimodal learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cancer has relational information residing at varying scales, modalities, and resolutions of the acquired data, such as radiology, pathology, genomics, proteomics, and clinical records. Integrating diverse data types can improve the accuracy and reliability of cancer diagnosis and treatment. There can be disease-related information that is too subtle for humans or existing technological tools to discern visually. Traditional methods typically focus on partial or unimodal information about biological systems at individual scales and fail to encapsulate the complete spectrum of the heterogeneous nature of data. Deep neural networks have facilitated the development of sophisticated multimodal data fusion approaches that can extract and integrate relevant information from multiple sources. Recent deep learning frameworks such as Graph Neural Networks (GNNs) and Transformers have shown remarkable success in multimodal learning. This review article provides an in-depth analysis of the state-of-the-art in GNNs and Transformers for multimodal data fusion in oncology settings, highlighting notable research studies and their findings. We also discuss the foundations of multimodal learning, inherent challenges, and opportunities for integrative learning in oncology. By examining the current state and potential future developments of multimodal data integration in oncology, we aim to demonstrate the promising role that multimodal neural networks can play in cancer prevention, early detection, and treatment through informed oncology practices in personalized settings.

Related papers

• Multimodal Data Integration for Precision Oncology: Challenges and Future Directions [10.817613081663007]
  The essence of precision oncology lies in its commitment to tailor targeted treatments and care measures to each patient based on the individual characteristics of the tumor. Over the past decade, multimodal data integration technology for precision oncology has made significant strides. We provide a comprehensive overview of about 300 papers detailing cutting-edge multimodal data integration techniques in precision oncology.
  arXiv  Detail & Related papers  (2024-06-28T02:35:05Z)
• HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data [10.774128925670183]
  This paper presents the Hybrid Early-fusion Attention Learning Network (HEALNet), a flexible multimodal fusion architecture. We conduct multimodal survival analysis on Whole Slide Images and Multi-omic data on four cancer datasets from The Cancer Genome Atlas (TCGA) HEALNet achieves state-of-the-art performance compared to other end-to-end trained fusion models.
  arXiv  Detail & Related papers  (2023-11-15T17:06:26Z)
• Building Flexible, Scalable, and Machine Learning-ready Multimodal Oncology Datasets [17.774341783844026]
  This work proposes Multimodal Integration of Oncology Data System (MINDS) MINDS is a flexible, scalable, and cost-effective metadata framework for efficiently fusing disparate data from public sources. By harmonizing multimodal data, MINDS aims to potentially empower researchers with greater analytical ability.
  arXiv  Detail & Related papers  (2023-09-30T15:44:39Z)
• Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
  We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
  arXiv  Detail & Related papers  (2023-05-25T16:29:16Z)
• Deep Biological Pathway Informed Pathology-Genomic Multimodal Survival Prediction [7.133948707208067]
  We propose PONET- a novel biological pathway-informed pathology-genomic deep model. Our proposed method achieves superior predictive performance and reveals meaningful biological interpretations.
  arXiv  Detail & Related papers  (2023-01-06T05:24:41Z)
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation [158.8192041981564]
  This paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance.
  arXiv  Detail & Related papers  (2022-01-07T07:46:01Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization [59.5104563755095]
  We introduce a simple but effective approach to improve the generalization capability of deep neural networks in the field of medical imaging classification. Motivated by the observation that the domain variability of the medical images is to some extent compact, we propose to learn a representative feature space through variational encoding.
  arXiv  Detail & Related papers  (2020-09-27T12:30:30Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data [75.73881040581767]
  We propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations. Our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.
  arXiv  Detail & Related papers  (2020-02-09T14:11:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.