Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data

• URL: http://arxiv.org/abs/2210.13889v2
• Date: Tue, 19 Sep 2023 09:40:15 GMT
• Title: Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data
• Authors: Huy Hoang Nguyen, Matthew B. Blaschko, Simo Saarakkala, Aleksei Tiulpin
• Abstract summary: We formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents, we predict prognosis with two transformer-based components. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer's disease clinical status directly from raw multi-modal data.
• Score: 13.766035805437847
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks are often applied to medical images to automate the problem of medical diagnosis. However, a more clinically relevant question that practitioners usually face is how to predict the future trajectory of a disease. Current methods for prognosis or disease trajectory forecasting often require domain knowledge and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents -- a radiologist and a general practitioner -- we predict prognosis with two transformer-based components that share information with each other. The first transformer in this framework aims to analyze the imaging data, and the second one leverages its internal states as inputs, also fusing them with auxiliary clinical data. The temporal nature of the problem is modeled within the transformer states, allowing us to treat the forecasting problem as a multi-task classification, for which we propose a novel loss. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer's disease clinical status directly from raw multi-modal data. The proposed method outperforms multiple state-of-the-art baselines with respect to performance and calibration, both of which are needed for real-world applications. An open-source implementation of our method is made publicly available at \url{https://github.com/Oulu-IMEDS/CLIMATv2}.

Related papers

• Time-aware Heterogeneous Graph Transformer with Adaptive Attention Merging for Health Event Prediction [6.578298085691462]
  We introduce a novel heterogeneous graph learning model designed to assimilate disease domain knowledge and elucidate the intricate relationships between drugs and diseases. When evaluated on two healthcare datasets, our approach demonstrated notable enhancements in both prediction accuracy and interpretability.
  arXiv  Detail & Related papers  (2024-04-23T08:01:30Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• Safe AI for health and beyond -- Monitoring to transform a health service [51.8524501805308]
  We will assess the infrastructure required to monitor the outputs of a machine learning algorithm. We will present two scenarios with examples of monitoring and updates of models.
  arXiv  Detail & Related papers  (2023-03-02T17:27:45Z)
• Sequential Diagnosis Prediction with Transformer and Ontological Representation [35.88195694025553]
  We propose an end-to-end robust transformer-based model called SETOR to handle irregular intervals between a patient's visits with admitted timestamps and length of stay in each visit. Experiments conducted on two real-world healthcare datasets show that, our sequential diagnoses prediction model SETOR achieves better predictive results than previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-09-07T13:09:55Z)
• Leveraging Clinical Context for User-Centered Explainability: A Diabetes Use Case [4.520155732176645]
  We implement a proof-of-concept (POC) in type-2 diabetes (T2DM) use case where we assess the risk of chronic kidney disease (CKD) Within the POC, we include risk prediction models for CKD, post-hoc explainers of the predictions, and other natural-language modules. Our POC approach covers multiple knowledge sources and clinical scenarios, blends knowledge to explain data and predictions to PCPs, and received an enthusiastic response from our medical expert.
  arXiv  Detail & Related papers  (2021-07-06T02:44:40Z)
• Semi-Supervised Variational Reasoning for Medical Dialogue Generation [70.838542865384]
  Two key characteristics are relevant for medical dialogue generation: patient states and physician actions. We propose an end-to-end variational reasoning approach to medical dialogue generation. A physician policy network composed of an action-classifier and two reasoning detectors is proposed for augmented reasoning ability.
  arXiv  Detail & Related papers  (2021-05-13T04:14:35Z)
• DeepProg: A Transformer-based Framework for Predicting Disease Prognosis [19.673447448533743]
  In this paper, we formulate the prognosis prediction task as a one-to-many sequence prediction problem. Inspired by a clinical decision making process with two agents -- a radiologist and a general practitioner, we propose a generic end-to-end transformer-based framework. The effectiveness and validation of the developed method are shown on synthetic data, and in the task of predicting the development of structural osteoarthritic changes in knee joints.
  arXiv  Detail & Related papers  (2021-04-08T09:53:18Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.