SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

• URL: http://arxiv.org/abs/2609.05174v1
• Date: Fri, 04 Sep 2026 14:13:24 GMT
• Title: SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis
• Abstract summary: Explainability is increasingly seen as a crucial requirement in AI-based medical diagnosis, particularly in safety-critical clinical decision-making.<n>This paper addresses the problem of self-explainable multimodal diagnosis by formulating it within the information bottleneck (IB) framework.<n>We propose a unified learning paradigm that jointly optimize predictive performance and modality-specific explainability.
• Score: 71.98702207668346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainability is increasingly seen as a crucial requirement in AI-based medical diagnosis, particularly in safety-critical clinical decision-making. Most existing explainability methods in healthcare operate in a post-hoc manner and are predominantly designed for unimodal data, which limits their applicability in increasingly prevalent multimodal diagnostic settings. This paper addresses the problem of self-explainable multimodal diagnosis by formulating it within the information bottleneck (IB) framework. We propose a unified learning paradigm that jointly optimizes predictive performance and modality-specific explainability by identifying the most informative elements inside each modality that contribute to diagnostic decisions. To enable tractable and stable optimization, we employ a matrix-based Renyi's $α$-order entropy functional under the assumption of sufficiently expressive encoders. Extensive experiments on representative medical datasets spanning heterogeneous modalities demonstrate that the proposed method consistently achieves strong diagnostic performance, including an absolute accuracy improvement of 9.1 percentage points on the iCTCF dataset. Moreover, the learned explanations provide transparent and modality-aware insights into feature relevance, thereby improving both the explainability and generalization.

Related papers

• Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases [59.5328455571863]
  ClinMM-Bench is the largest multi-turn multimodal clinical diagnostic evaluation benchmark to date.<n>It contains 1,089 challenging real-world clinical cases and 3,760 medical images across eight specialties.<n> proprietary models achieved the highest overall diagnostic accuracy, but the proportion of completely correct diagnoses remained limited.
  arXiv  Detail & Related papers  (2026-07-28T16:19:03Z)
• Timely Clinical Diagnosis through Active Test Selection [49.091903570068155]
  We propose ACTMED (Adaptive Clinical Test selection via Model-based Experimental Design) to better emulate real-world diagnostic reasoning.<n>LLMs act as flexible simulators, generating plausible patient state distributions and supporting belief updates without requiring structured, task-specific training data.<n>We evaluate ACTMED on real-world datasets and show it can optimize test selection to improve diagnostic accuracy, interpretability, and resource use.
  arXiv  Detail & Related papers  (2025-10-21T18:10:45Z)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis [56.373297358647655]
  Retrieval-Augmented Diagnosis (RAD) is a novel framework that injects external knowledge into multimodal models directly on downstream tasks.<n>RAD operates through three key mechanisms: retrieval and refinement of disease-centered knowledge from multiple medical sources, a guideline-enhanced contrastive loss transformer, and a dual decoder.
  arXiv  Detail & Related papers  (2025-09-24T10:36:14Z)
• Multimodal Attention-Aware Fusion for Diagnosing Distal Myopathy: Evaluating Model Interpretability and Clinician Trust [19.107204920543676]
  Distal myopathy represents a heterogeneous group of skeletal muscle disorders with broad clinical manifestations.<n>We propose a novel multimodal attention-aware fusion architecture that combines features extracted from two distinct deep learning models.<n>Our approach integrates these features through an attention gate mechanism, enhancing both predictive performance and interpretability.
  arXiv  Detail & Related papers  (2025-08-02T11:08:55Z)
• LLM-Driven Medical Document Analysis: Enhancing Trustworthy Pathology and Differential Diagnosis [13.435898630240416]
  We propose a trustworthy medical document analysis platform that fine-tunes a LLaMA-v3 using low-rank adaptation.<n>Our approach utilizes DDXPlus, the largest benchmark dataset for differential diagnosis.<n>The developed web-based platform allows users to submit their own unstructured medical documents and receive accurate, explainable diagnostic results.
  arXiv  Detail & Related papers  (2025-06-24T15:12:42Z)
• GEMeX-RMCoT: An Enhanced Med-VQA Dataset for Region-Aware Multimodal Chain-of-Thought Reasoning [60.03671205298294]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability.<n>This work first proposes a Region-Aware Multimodal Chain-of-Thought dataset, in which the process of producing an answer is preceded by a sequence of intermediate reasoning steps.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• Empowering Medical Multi-Agents with Clinical Consultation Flow for Dynamic Diagnosis [20.59719567178192]
  We propose a multi-agent framework inspired by consultation flow and reinforcement learning (RL) to simulate the entire consultation process.<n>Our approach incorporates a hierarchical action set, structured from clinic consultation flow and medical textbook, to effectively guide the decision-making process.<n>This strategy improves agent interactions, enabling them to adapt and optimize actions based on the dynamic state.
  arXiv  Detail & Related papers  (2025-03-19T08:47:18Z)
• XAI for In-hospital Mortality Prediction via Multimodal ICU Data [57.73357047856416]
  We propose an efficient, explainable AI solution for predicting in-hospital mortality via multimodal ICU data. We employ multimodal learning in our framework, which can receive heterogeneous inputs from clinical data and make decisions. Our framework can be easily transferred to other clinical tasks, which facilitates the discovery of crucial factors in healthcare research.
  arXiv  Detail & Related papers  (2023-12-29T14:28:04Z)
• Multimodal Explainability via Latent Shift applied to COVID-19 stratification [0.7831774233149619]
  We present a deep architecture, which jointly learns modality reconstructions and sample classifications. We validate our approach in the context of COVID-19 pandemic using the AIforCOVID dataset.
  arXiv  Detail & Related papers  (2022-12-28T20:07:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.