Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multi-Cancer Cohort

• URL: http://arxiv.org/abs/2603.29793v1
• Date: Tue, 31 Mar 2026 14:27:21 GMT
• Title: Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multi-Cancer Cohort
• Abstract summary: Multimodal Machine Learning offers a holistic view of a patient's status, integrating structured and unstructured data from electronic health records.<n>We propose a framework to predict metastasis risk one month prior to diagnosis, using six months of clinical history from EHR data.
• Score: 1.5829605001132387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Machine Learning offers a holistic view of a patient's status, integrating structured and unstructured data from electronic health records (EHR). We propose a framework to predict metastasis risk one month prior to diagnosis, using six months of clinical history from EHR data. Data from four cancer cohorts collected at Karolinska University Hospital (Stockholm, Sweden) were analyzed: breast (n = 743), colon (n = 387), lung (n = 870), and prostate (n = 1890). The dataset included demographics, comorbidities, laboratory results, medications, and clinical text. We compared traditional and deep learning classifiers across single modalities and multimodal combinations, using various fusion strategies and a Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) 2a design, with an 80-20 development-validation split to ensure a rigorous, repeatable evaluation. Performance was evaluated using AUROC, AUPRC, F1 score, sensitivity, and specificity. We then employed a multimodal adaptation of SHAP to analyze the classifiers' reasoning. Intermediate fusion achieved the highest F1 scores on breast (0.845), colon (0.786), and prostate cancer (0.845), demonstrating strong predictive performance. For lung cancer, the intermediate fusion achieved an F1 score of 0.819, while the text-only model achieved the highest, with an F1 score of 0.829. Deep learning classifiers consistently outperformed traditional models. Colon cancer, the smallest cohort, had the lowest performance, highlighting the importance of sufficient training data. SHAP analysis showed that the relative importance of modalities varied across cancer types. Fusion strategies offer distinct strengths and weaknesses. Intermediate fusion consistently delivered the best results, but strategy choices should align with data characteristics and organizational needs.

Related papers

• Trusting Right Predictions for Wrong Reasons: A LIME Based Analysis of Deep Learning Interpretability in Lung Cancer Diagnosis [0.0]
  Lung cancer is the leading cause of cancer-related mortality, with approximately 2.5 million new cases and 1.8 million deaths annually.<n> evaluation has largely focused on predictive accuracy, leaving their decision-making processes insufficiently examined.<n>This study compares three models: a Convolutional Neural Network (CNN), a pretrained ResNet50, and a Vision Transformer (ViT), trained on the IQ-OTH/NCCD lung cancer CT dataset.<n>All three models achieved strong classification performance, with ResNet50 attaining 98.61% accuracy, CNN 97.91%, and ViT 93.75%, while all achieved
  arXiv  Detail & Related papers  (2026-06-14T22:08:31Z)
• Multimodal Fusion of Histopathology Images and Electronic Health Records for Early Breast Cancer Diagnosis [15.70912899013153]
  This paper presents a systematic framework that integrates patch-level histopathology features with structured clinical data.<n>We train and evaluate unimodal image models (a simple CNN baseline and ResNet-18 with transfer learning) and intermediate-fusion models (XGBoost and a multilayer perceptron)<n>Our results demonstrate that multimodal integration delivers meaningful improvements in both predictive performance and clinical transparency.
  arXiv  Detail & Related papers  (2026-04-18T19:46:34Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer [54.58205672910646]
  RenalCLIP is a visual-language foundation model for characterization, diagnosis and prognosis of renal mass.<n>It achieved better performance and superior generalizability across 10 core tasks spanning the full clinical workflow of kidney cancer.
  arXiv  Detail & Related papers  (2025-08-22T17:48:19Z)
• Automatic Cough Analysis for Non-Small Cell Lung Cancer Detection [33.37223681850477]
  Early detection of non-small cell lung cancer (NSCLC) is critical for improving patient outcomes.<n>We explore the use of automatic cough analysis as a pre-screening tool for distinguishing between NSCLC patients and healthy controls.<n>Recordings were analyzed using machine learning techniques, such as support vector machine (SVM) and XGBoost.
  arXiv  Detail & Related papers  (2025-07-25T11:30:22Z)
• Machine learning-based multimodal prognostic models integrating pathology images and high-throughput omic data for overall survival prediction in cancer: a systematic review [0.59374762912328]
  Multimodal machine learning integrating histopathology and molecular data shows promise for cancer prognostication.<n>We systematically reviewed studies combining whole slide images (WSIs) and high- throughput omics to predict overall survival.<n>All studies showed unclear/high bias, limited external validation, and little focus on clinical utility.
  arXiv  Detail & Related papers  (2025-07-22T11:02:51Z)
• Predicting Length of Stay in Neurological ICU Patients Using Classical Machine Learning and Neural Network Models: A Benchmark Study on MIMIC-IV [49.1574468325115]
  This study explores multiple ML approaches for predicting LOS in ICU specifically for the patients with neurological diseases based on the MIMIC-IV dataset.<n>The evaluated models include classic ML algorithms (K-Nearest Neighbors, Random Forest, XGBoost and CatBoost) and Neural Networks (LSTM, BERT and Temporal Fusion Transformer)
  arXiv  Detail & Related papers  (2025-05-23T14:06:42Z)
• A multimodal ensemble approach for clear cell renal cell carcinoma treatment outcome prediction [6.199310532720352]
  We developed a multi-modal ensemble model (MMEM) that integrates clinical data, multi-omics data, and histopathology whole slide image (WSI) data.<n>MMEM predicted overall survival (OS) and disease-free survival (DFS) for ccRCC patients.
  arXiv  Detail & Related papers  (2024-12-10T02:51:14Z)
• HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models [11.03503711055749]
  HONeYBEE is an open-source framework that integrates multimodal biomedical data for oncology applications.<n>It processes clinical data (structured and unstructured), whole-slide images, radiology scans, and molecular profiles to generate unified patient-level embeddings.<n>These embeddings enable survival prediction, cancer-type classification, patient similarity retrieval, and cohort clustering.
  arXiv  Detail & Related papers  (2024-05-13T04:35:14Z)
• A Meta-GNN approach to personalized seizure detection and classification [53.906130332172324]
  We propose a personalized seizure detection and classification framework that quickly adapts to a specific patient from limited seizure samples. We train a Meta-GNN based classifier that learns a global model from a set of training patients. We show that our method outperforms the baselines by reaching 82.7% on accuracy and 82.08% on F1 score after only 20 iterations on new unseen patients.
  arXiv  Detail & Related papers  (2022-11-01T14:12:58Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.