Whole-Body MRI Classification via Prompt-Based Clinical Conditioning

• URL: http://arxiv.org/abs/2608.30824v1
• Date: Mon, 31 Aug 2026 14:03:41 GMT
• Title: Whole-Body MRI Classification via Prompt-Based Clinical Conditioning
• Abstract summary: TACTIC (Tabular-Attribute Conditioned Transformer for Image Classification) is a prompt-based framework that integrates WB-MRI and structured clinical data through conditional visual feature learning.<n>We evaluate TACTIC on five WB-MRI classification tasks spanning systemic and oncologic applications, including diabetes,COPD, breast cancer, prostate cancer, and metastasis diagnosis.
• Score: 3.7441840033323266
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Combining whole-body magnetic resonance imaging (WB-MRI) with clinical variables has the potential to improve systemic disease diagnosis by leveraging complementary sources of patient information. However, structured clinical variables are often incomplete or missing, limiting the applicability of conventional multimodal fusion methods that assume fixed inputs. In this work, we propose TACTIC (Tabular-Attribute Conditioned Transformer for Image Classification), a prompt-based multimodal framework that integrates WB-MRI and structured clinical data through conditional visual feature learning. By encoding clinical attributes as prompts, TACTIC supports an arbitrary number of tabular inputs and naturally handles missing data without requiring imputation or fixed input structures. We evaluate TACTIC on five WB-MRI classification tasks spanning systemic and oncologic applications, including diabetes, chronic obstructive pulmonary disease (COPD), breast cancer, prostate cancer, and metastasis diagnosis. Across all tasks, TACTIC consistently improves performance over image-only baselines when clinical information is available while maintaining strong predictive capability under incomplete tabular inputs. Our results demonstrate the effectiveness of prompt-based models as a flexible approach for improving WB-MRI analysis using clinical context. The model weights and code are available at https://github.com/lauradaza/TACTIC

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• JI-ADF: Joint-Individual Learning with Adaptive Decision Fusion for Multimodal Skin Lesion Classification [1.57333503756856]
  Skin lesion classification is essential for early dermatological diagnosis.<n>Many existing computer-aided systems rely primarily on dermoscopic images and underutilize the multimodal evidence.<n>We propose textbfJI-ADF, a trimodal deep learning framework that integrates dermoscopic images, clinical photographs, and structured patient metadata.
  arXiv  Detail & Related papers  (2026-04-30T02:48:15Z)
• ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning [58.01333341218153]
  We propose ClinDEF, a dynamic framework for assessing clinical reasoning in LLMs through simulated diagnostic dialogues.<n>Our method generates patient cases and facilitates multi-turn interactions between an LLM-based doctor and an automated patient agent.<n>Experiments show that ClinDEF effectively exposes critical clinical reasoning gaps in state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-12-29T12:58:58Z)
• Prompt-Aware Adaptive Elastic Weight Consolidation for Continual Learning in Medical Vision-Language Models [0.0]
  Medical vision-language models must preserve cross-modal alignments between medical images and clinical terminology.<n>We introduce Prompt- Aware Adaptive Elastic Weight Consolidation (PA-EWC), a novel continual learning approach.<n> Experimental results demonstrate that PA-EWC reduces catastrophic forgetting by up to 17.58% compared to baseline methods.
  arXiv  Detail & Related papers  (2025-11-25T12:22:56Z)
• Timely Clinical Diagnosis through Active Test Selection [49.091903570068155]
  We propose ACTMED (Adaptive Clinical Test selection via Model-based Experimental Design) to better emulate real-world diagnostic reasoning.<n>LLMs act as flexible simulators, generating plausible patient state distributions and supporting belief updates without requiring structured, task-specific training data.<n>We evaluate ACTMED on real-world datasets and show it can optimize test selection to improve diagnostic accuracy, interpretability, and resource use.
  arXiv  Detail & Related papers  (2025-10-21T18:10:45Z)
• impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction [75.43342771863837]
  We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy.<n>It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches.<n>Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets.
  arXiv  Detail & Related papers  (2025-08-08T10:01:16Z)
• NeuroMoE: A Transformer-Based Mixture-of-Experts Framework for Multi-Modal Neurological Disorder Classification [3.5313393560458826]
  Deep Learning has recently emerged as a powerful tool for extracting meaningful patterns from medical data to aid in diagnosis.<n>We propose a novel transformer-based Mixture-of-Experts (MoE) framework for classifying neurological disorders.<n>Our framework achieves a validation accuracy of 82.47%, outperforming baseline methods by over 10%.
  arXiv  Detail & Related papers  (2025-06-17T20:40:06Z)
• Towards Accurate and Interpretable Neuroblastoma Diagnosis via Contrastive Multi-scale Pathological Image Analysis [16.268045905735818]
  We propose CMSwinKAN, a contrastive-learning-based multi-scale feature fusion model tailored for pathological image classification.<n>By fusing multi-scale features and leveraging contrastive learning strategies, CMSwinKAN mimics clinicians' comprehensive approach.<n>Results demonstrate that CMSwinKAN performs better than existing state-of-the-art pathology-specific models pre-trained on large datasets.
  arXiv  Detail & Related papers  (2025-04-18T15:39:46Z)
• Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy [63.39037092484374]
  Synthetic Data Generation based on Artificial Intelligence (AI) can transform the way clinical medicine is delivered.<n>This study focuses on the clinical evaluation of medical SDG, with a proof-of-concept investigation on diagnosing Inflammatory Bowel Disease (IBD) using Wireless Capsule Endoscopy (WCE) images.<n>The results show that TIDE-II generates clinically plausible, very realistic WCE images, of improved quality compared to relevant state-of-the-art generative models.
  arXiv  Detail & Related papers  (2024-10-31T19:48:50Z)
• DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification [8.95697172316912]
  We propose a Dynamic Screening and Clinical-Enhanced Network (DSCENet) for the subtype classification of MPNs on the multimodal fusion of whole slide images (WSIs) and clinical information. Our approach has been validated on the real clinical data, achieving an increase of 7.91% AUC and 16.89% accuracy compared with the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-11T04:13:58Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics [63.106382317917344]
  We report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases.
  arXiv  Detail & Related papers  (2023-06-01T16:23:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.