Feature importance to explain multimodal prediction models. A clinical use case

• URL: http://arxiv.org/abs/2404.18631v1
• Date: Mon, 29 Apr 2024 12:11:26 GMT
• Title: Feature importance to explain multimodal prediction models. A clinical use case
• Authors: Jorn-Jan van de Beld, Shreyasi Pathak, Jeroen Geerdink, Johannes H. Hegeman, Christin Seifert,
• Abstract summary: Surgery to treat elderly hip fracture patients may cause complications that can lead to early mortality. We develop a multimodal deep-learning model for post-operative mortality prediction using pre-operative and per-operative data.
• Score: 2.2527696312643637
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Surgery to treat elderly hip fracture patients may cause complications that can lead to early mortality. An early warning system for complications could provoke clinicians to monitor high-risk patients more carefully and address potential complications early, or inform the patient. In this work, we develop a multimodal deep-learning model for post-operative mortality prediction using pre-operative and per-operative data from elderly hip fracture patients. Specifically, we include static patient data, hip and chest images before surgery in pre-operative data, vital signals, and medications administered during surgery in per-operative data. We extract features from image modalities using ResNet and from vital signals using LSTM. Explainable model outcomes are essential for clinical applicability, therefore we compute Shapley values to explain the predictions of our multimodal black box model. We find that i) Shapley values can be used to estimate the relative contribution of each modality both locally and globally, and ii) a modified version of the chain rule can be used to propagate Shapley values through a sequence of models supporting interpretable local explanations. Our findings imply that a multimodal combination of black box models can be explained by propagating Shapley values through the model sequence.

Related papers

• Continually Evolved Multimodal Foundation Models for Cancer Prognosis [50.43145292874533]
  Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates. Previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information. Existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals. Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities.
  arXiv  Detail & Related papers  (2025-01-30T06:49:57Z)
• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Scaling by training on large datasets has been shown to enhance the quality and fidelity of image generation and manipulation with diffusion models. Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation. Our results demonstrate significant performance gains in various scenarios when combined with different fine-tuning schemes.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• Enhanced Survival Prediction in Head and Neck Cancer Using Convolutional Block Attention and Multimodal Data Fusion [7.252280210331731]
  This paper proposes a deep learning-based approach to predict survival outcomes in head and neck cancer patients. Our method integrates feature extraction with a Convolutional Block Attention Module (CBAM) and a multi-modal data fusion layer. The final prediction is achieved through a fully parametric discrete-time survival model.
  arXiv  Detail & Related papers  (2024-10-29T07:56:04Z)
• Survival Prediction in Lung Cancer through Multi-Modal Representation Learning [9.403446155541346]
  This paper presents a novel approach to survival prediction by harnessing comprehensive information from CT and PET scans, along with associated Genomic data. We aim to develop a robust predictive model for survival outcomes by integrating multi-modal imaging data with genetic information.
  arXiv  Detail & Related papers  (2024-09-30T10:42:20Z)
• Prediction of Post-Operative Renal and Pulmonary Complications Using Transformers [69.81176740997175]
  We evaluate the performance of transformer-based models in predicting postoperative acute renal failure, pulmonary complications, and postoperative in-hospital mortality. Our results demonstrate that transformer-based models can achieve superior performance in predicting postoperative complications and outperform traditional machine learning models.
  arXiv  Detail & Related papers  (2023-06-01T14:08:05Z)
• Self-explaining Hierarchical Model for Intraoperative Time Series [6.877686657275981]
  We propose a hierarchical model combining the strength of both attention and recurrent models for intraoperative time series. Experiments on a large dataset of 111,888 surgeries with multiple outcomes and an external high-resolution ICU dataset show that our model can achieve strong predictive performance.
  arXiv  Detail & Related papers  (2022-10-10T03:24:18Z)
• Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation [7.2666838978096875]
  Existing approaches typically train models in a patient-specific fashion due to the highly personalized characteristics of epileptic signals. A patient-specific model can then be obtained with the help of distilled knowledge and additional personalized data. Five state-of-the-art seizure prediction methods are trained on the CHB-MIT sEEG database with our proposed scheme.
  arXiv  Detail & Related papers  (2022-02-25T10:30:29Z)
• Multi-task fusion for improving mammography screening data classification [3.7683182861690843]
  We propose a pipeline approach, where we first train a set of individual, task-specific models. We then investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. Our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling.
  arXiv  Detail & Related papers  (2021-12-01T13:56:27Z)
• Approximate Bayesian Computation for an Explicit-Duration Hidden Markov Model of COVID-19 Hospital Trajectories [55.786207368853084]
  We address the problem of modeling constrained hospital resources in the midst of the COVID-19 pandemic. For broad applicability, we focus on the common yet challenging scenario where patient-level data for a region of interest are not available. We propose an aggregate count explicit-duration hidden Markov model, nicknamed the ACED-HMM, with an interpretable, compact parameterization.
  arXiv  Detail & Related papers  (2021-04-28T15:32:42Z)
• A General Framework for Survival Analysis and Multi-State Modelling [70.31153478610229]
  We use neural ordinary differential equations as a flexible and general method for estimating multi-state survival models. We show that our model exhibits state-of-the-art performance on popular survival data sets and demonstrate its efficacy in a multi-state setting.
  arXiv  Detail & Related papers  (2020-06-08T19:24:54Z)
• M2Net: Multi-modal Multi-channel Network for Overall Survival Time Prediction of Brain Tumor Patients [151.4352001822956]
  Early and accurate prediction of overall survival (OS) time can help to obtain better treatment planning for brain tumor patients. Existing prediction methods rely on radiomic features at the local lesion area of a magnetic resonance (MR) volume. We propose an end-to-end OS time prediction model; namely, Multi-modal Multi-channel Network (M2Net)
  arXiv  Detail & Related papers  (2020-06-01T05:21:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.