Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis

• URL: http://arxiv.org/abs/2210.14377v1
• Date: Tue, 25 Oct 2022 23:03:05 GMT
• Title: Fusing Modalities by Multiplexed Graph Neural Networks for Outcome Prediction in Tuberculosis
• Authors: Niharika S. D'Souza, Hongzhi Wang, Andrea Giovannini, Antonio Foncubierta-Rodriguez, Kristen L. Beck, Orest Boyko, and Tanveer Syeda-Mahmood
• Abstract summary: We present a novel fusion framework using multiplexed graphs and derive a new graph neural network for learning from such graphs. We present results that show that our proposed method outperforms state-of-the-art methods of fusing modalities for multi-outcome prediction on a large Tuberculosis (TB) dataset.
• Score: 3.131872070347212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In a complex disease such as tuberculosis, the evidence for the disease and its evolution may be present in multiple modalities such as clinical, genomic, or imaging data. Effective patient-tailored outcome prediction and therapeutic guidance will require fusing evidence from these modalities. Such multimodal fusion is difficult since the evidence for the disease may not be uniform across all modalities, not all modality features may be relevant, or not all modalities may be present for all patients. All these nuances make simple methods of early, late, or intermediate fusion of features inadequate for outcome prediction. In this paper, we present a novel fusion framework using multiplexed graphs and derive a new graph neural network for learning from such graphs. Specifically, the framework allows modalities to be represented through their targeted encodings, and models their relationship explicitly via multiplexed graphs derived from salient features in a combined latent space. We present results that show that our proposed method outperforms state-of-the-art methods of fusing modalities for multi-outcome prediction on a large Tuberculosis (TB) dataset.

Related papers

• GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation [68.63955715643974]
  Modality-prompted Heterogeneous Graph for Omnimodal Learning (GTP-4o) We propose an innovative Modality-prompted Heterogeneous Graph for Omnimodal Learning (GTP-4o)
  arXiv  Detail & Related papers  (2024-07-08T01:06:13Z)
• DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency [18.291267748113142]
  We propose DrFuse to achieve effective clinical multi-modal fusion. We address the missing modality issue by disentangling the features shared across modalities and those unique within each modality. We validate the proposed method using real-world large-scale datasets, MIMIC-IV and MIMIC-CXR.
  arXiv  Detail & Related papers  (2024-03-10T12:41:34Z)
• MaxCorrMGNN: A Multi-Graph Neural Network Framework for Generalized Multimodal Fusion of Medical Data for Outcome Prediction [3.2889220522843625]
  We develop an innovative fusion approach called MaxCorr MGNN that models non-linear modality correlations within and across patients. We then design, for the first time, a generalized multi-layered graph neural network (MGNN) for task-informed reasoning in multi-layered graphs. We evaluate our model an outcome prediction task on a Tuberculosis dataset consistently outperforming several state-of-the-art neural, graph-based and traditional fusion techniques.
  arXiv  Detail & Related papers  (2023-07-13T23:52:41Z)
• Incomplete Multimodal Learning for Complex Brain Disorders Prediction [65.95783479249745]
  We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative cohort.
  arXiv  Detail & Related papers  (2023-05-25T16:29:16Z)
• Bayesian Networks for the robust and unbiased prediction of depression and its symptoms utilizing speech and multimodal data [65.28160163774274]
  We apply a Bayesian framework to capture the relationships between depression, depression symptoms, and features derived from speech, facial expression and cognitive game data collected at thymia.
  arXiv  Detail & Related papers  (2022-11-09T14:48:13Z)
• Multi-modal Graph Learning for Disease Prediction [35.156975779372836]
  We propose an end-to-end Multi-modal Graph Learning framework (MMGL) for disease prediction with multi-modality. Instead of defining the graph manually, the latent graph structure is captured through an effective way of adaptive graph learning. An extensive group of experiments on two disease prediction tasks demonstrates that the proposed MMGL achieves more favorable performance.
  arXiv  Detail & Related papers  (2022-03-11T12:33:20Z)
• Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer's Disease Prediction [30.99183477161096]
  We propose a novel representation learning and adversarial hypergraph fusion framework for Alzheimer's disease diagnosis. Our model achieves superior performance on Alzheimer's disease detection compared with other related models.
  arXiv  Detail & Related papers  (2021-07-21T08:08:05Z)
• Multi-modal Graph Learning for Disease Prediction [35.4310911850558]
  We propose an end-to-end Multimodal Graph Learning framework (MMGL) for disease prediction. Instead of defining the adjacency matrix manually as existing methods, the latent graph structure can be captured through a novel way of adaptive graph learning.
  arXiv  Detail & Related papers  (2021-07-01T03:59:22Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Dynamic Graph Correlation Learning for Disease Diagnosis with Incomplete Labels [66.57101219176275]
  Disease diagnosis on chest X-ray images is a challenging multi-label classification task. We propose a Disease Diagnosis Graph Convolutional Network (DD-GCN) that presents a novel view of investigating the inter-dependency among different diseases. Our method is the first to build a graph over the feature maps with a dynamic adjacency matrix for correlation learning.
  arXiv  Detail & Related papers  (2020-02-26T17:10:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.