Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

• URL: http://arxiv.org/abs/2607.09982v1
• Date: Fri, 10 Jul 2026 21:15:53 GMT
• Title: Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction
• Abstract summary: We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning.<n>Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions.<n>We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV.
• Score: 4.160679255824342
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance. However, most existing approaches rely on deep fusion, which obscures how individual modalities contribute to predictions and limits the interpretability of multimodal reasoning. We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning across three EHR modalities: structured longitudinal variables (L), clinical notes (N), and chest X-rays (I). Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions. To audit multimodal reasoning and assess robustness, we introduce inference-time route masking, which simulates missing modalities and reweights the remaining routes without retraining. We analyze changes in performance and routing weights under these scenarios to understand model decision-making. We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV, revealing systematic differences in modality reliance across clinical condition groups. Overall, our framework offers a transparent, auditable, and practical approach to multimodal clinical prediction, providing interpretability, robustness, and insights into how different data sources drive model decisions.

Related papers

• DAIF: A Data-Driven Intermediate Fusion Framework for Multimodal Supervised Learning via Approximate Message Passing [11.222071637855079]
  DAIF is a data adaptive intermediate fusion framework to learn fusion structure directly from data.<n>We operate under a Bayesian multimodal factor model where the prior on the latent factors determines the cross-modal dependence.<n>Our method clusters modalities based on estimated intermodal dependence, then performs clusterwise empirical Bayes estimation of the priors.
  arXiv  Detail & Related papers  (2026-08-03T18:10:58Z)
• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• MMEB-V3: Measuring the Performance Gaps of Omni-Modality Embedding Models [70.34265674686516]
  Multimodal embedding models aim to map heterogeneous inputs, such as text, images, videos, and audio, into a shared semantic space.<n>We introduce MMEB-V3, a benchmark that evaluates embeddings across text, image, video, audio, as well as agent-centric scenarios.<n>We conduct a systematic analysis of full-modality embeddings and identify three key findings.
  arXiv  Detail & Related papers  (2026-04-25T14:15:05Z)
• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• Post-TIPS Prediction via Multimodal Interaction: A Multi-Center Dataset and Framework for Survival, Complication, and Portal Pressure Assessment [23.819390753301136]
  Transjugular intrahepatic portosystemic shunt (TIPS) is an established procedure for portal hypertension, but provides variable survival and frequent overt hepatic encephalopathy (OHE)<n>Current studies typically build machine learning models from preoperative CT images or clinical characteristics.<n>We present MultiTIPS, the first public multi-center dataset for TIPS prognosis, and propose a novel multimodal prognostic framework based on it.
  arXiv  Detail & Related papers  (2025-10-12T06:11:59Z)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis [56.373297358647655]
  Retrieval-Augmented Diagnosis (RAD) is a novel framework that injects external knowledge into multimodal models directly on downstream tasks.<n>RAD operates through three key mechanisms: retrieval and refinement of disease-centered knowledge from multiple medical sources, a guideline-enhanced contrastive loss transformer, and a dual decoder.
  arXiv  Detail & Related papers  (2025-09-24T10:36:14Z)
• A Comparative Study of Controllability, Explainability, and Performance in Dysfluency Detection Models [6.837099592935974]
  We compare four dysfluency modeling approaches: YOLO-Stutter, FluentNet, UDM, and SSDM.<n>YOLO-Stutter and FluentNet provide efficiency and simplicity, but with limited transparency.<n>UDM achieves the best balance of accuracy and clinical interpretability.
  arXiv  Detail & Related papers  (2025-08-25T14:23:09Z)
• Continually Evolved Multimodal Foundation Models for Cancer Prognosis [50.43145292874533]
  Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates.<n>Previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information.<n>Existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals.<n>Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities.
  arXiv  Detail & Related papers  (2025-01-30T06:49:57Z)
• Multimodal Clinical Trial Outcome Prediction with Large Language Models [28.95412904299012]
  We propose a multimodal mixture-of-experts (LIFTED) approach for clinical trial outcome prediction.<n>LIFTED unifies different modality data by transforming them into natural language descriptions.<n>Then, LIFTED constructs unified noise-resilient encoders to extract information from modal-specific language descriptions.
  arXiv  Detail & Related papers  (2024-02-09T16:18:38Z)
• XAI for In-hospital Mortality Prediction via Multimodal ICU Data [57.73357047856416]
  We propose an efficient, explainable AI solution for predicting in-hospital mortality via multimodal ICU data. We employ multimodal learning in our framework, which can receive heterogeneous inputs from clinical data and make decisions. Our framework can be easily transferred to other clinical tasks, which facilitates the discovery of crucial factors in healthcare research.
  arXiv  Detail & Related papers  (2023-12-29T14:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.