A Discordance-Aware Multimodal Framework with Multi-Agent Clinical Reasoning

• URL: http://arxiv.org/abs/2604.16333v1
• Date: Thu, 12 Mar 2026 21:07:11 GMT
• Title: A Discordance-Aware Multimodal Framework with Multi-Agent Clinical Reasoning
• Abstract summary: Knee osteoarthritis frequently exhibits discordance between structural damage observed in imaging and patient-reported symptoms such as pain.<n>We propose a discordance aware multimodal framework that combines machine learning prediction models with a tool grounded multi agent reasoning system.
• Score: 5.675348355722554
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knee osteoarthritis frequently exhibits discordance between structural damage observed in imaging and patient-reported symptoms such as pain. This mismatch complicates clinical interpretation and patient stratification and remains insufficiently modeled in existing decision support systems. We propose a discordance aware multimodal framework that combines machine learning prediction models with a tool grounded multi agent reasoning system. Using baseline data from the FNIH Osteoarthritis Biomarkers Consortium, we trained multimodal models to predict two progression tasks, joint space loss only progression versus non progression, and pain only progression versus non progression. The predictive system integrates three modality specific experts: a CatBoost tabular model using demographic, radiographic, MRI-derived scalar, and biomarker features; MRI image embeddings extracted using a ResNet18 backbone; and Xray embeddings derived from the same architecture. Expert predictions are fused using a stacking ensemble. Residual based models estimate expected pain from structural features, enabling the computation of a pain structure discordance score between observed and expected symptoms. A multi-agent reasoning layer interprets these signals to assign clinically interpretable OA phenotypes and generate phenotype specific management recommendations.

Related papers

• Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation [66.62662330558906]
  CORAL is a multimodal framework that integrates spatial grounding and concept-level supervision into a unified reasoning process.<n>CORAL employs a prompt-driven medical segmentation model to localize lesions and predicts multi-class clinical attributes through a Concept Bottleneck module.<n>Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs.
  arXiv  Detail & Related papers  (2026-09-14T10:21:23Z)
• Motion-Guided Causal Disentanglement for Robust Multi-View Cine Cardiac MRI Diagnosis [0.4638331367650009]
  We propose a Motion-Guided View-Disease Disentanglement framework MoViD built upon a ViT-MAE backbone.<n>We evaluate the framework on a private clinical venous thrombosis dataset and two public benchmarks.
  arXiv  Detail & Related papers  (2026-06-03T03:45:11Z)
• Multimodal Graph-based Classification of Esophageal Motility Disorders [73.90451172929117]
  Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry data and variability in clinical interpretation.<n>This work explores the feasibility of a multimodal Machine Learning-based classification approach that combines HRIM recordings with patient-specific information and incorporates a graph-based modeling of esophageal physiology.
  arXiv  Detail & Related papers  (2026-05-13T14:52:12Z)
• Toward Personalized Digital Twins for Cognitive Decline Assessment: A Multimodal, Uncertainty-Aware Framework [0.2566571621858396]
  The PCD-DT framework provides a principled starting point for personalized in silico modeling in neurodegenerative disease.<n>This work positions PCD-DT as a foundational step toward clinically deployable, uncertainty-aware digital twin systems.
  arXiv  Detail & Related papers  (2026-04-29T21:40:55Z)
• Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling [24.896312231457653]
  We develop a framework to profile and handle missing modalities while addressing the canonical desideratum of safe, transparent clinical AI.<n>We show that autoregressive sequence modeling with transformer-based architectures outperforms baselines on the MIMIC-IV and eICU fine-tuning benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T18:55:56Z)
• Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models [56.29123284262618]
  This work proposes leveraging the universal representation capabilities of visual foundation models to estimate inherent data uncertainty.<n>We analyze the feature diversity of the model's decoded representations and quantify their singular value energy to define the semantic perception scale for each class.<n>Based on this foundation, we design two uncertainty-driven application strategies: (1) the aleatoric uncertainty-aware data filtering mechanism to eliminate potentially noisy samples and enhance model learning quality; and (2) the dynamic uncertainty-aware optimization strategy that adaptively adjusts class-specific loss weights during training based on the semantic perception scale, combined with a label denoising mechanism to improve training stability
  arXiv  Detail & Related papers  (2026-04-13T03:59:54Z)
• A multimodal vision foundation model for generalizable knee pathology [40.03838145472935]
  Musculoskeletal disorders represent an urgent demand for precise interpretation of medical imaging.<n>Current artificial intelligence approaches in orthopedics rely on task-specific, supervised learning paradigms.<n>We introduce OrthoFoundation, a multimodal vision foundation model optimized for musculoskeletal pathology.
  arXiv  Detail & Related papers  (2026-01-26T08:14:51Z)
• Clinical NLP with Attention-Based Deep Learning for Multi-Disease Prediction [44.0876796031468]
  This paper addresses the challenges posed by the unstructured nature and high-dimensional semantic complexity of electronic health record texts.<n>A deep learning method based on attention mechanisms is proposed to achieve unified modeling for information extraction and multi-label disease prediction.
  arXiv  Detail & Related papers  (2025-07-02T07:45:22Z)
• Going beyond explainability in multi-modal stroke outcome prediction models [0.0]
  We adapt the xAI methods Grad-CAM and Occlusion to multi-modal, partly interpretable deep transformation models (dTMs)<n>The dTMs achieve state-of-the-art prediction performance, with area under the curve (AUC) values close to 0.8.<n>Explanation maps calculated from brain imaging dTMs for functional outcome highlighted critical brain regions such as the frontal lobe.
  arXiv  Detail & Related papers  (2025-04-07T09:56:16Z)
• Patch-based Intuitive Multimodal Prototypes Network (PIMPNet) for Alzheimer's Disease classification [3.144057505325736]
  Part-prototype neural networks integrate the computational advantages of Deep Learning (DL) in an interpretable-by-design architecture. We present PIMPNet, the first interpretable multimodal model for 3D images and demographics applied to the binary classification of Alzheimer's Disease (AD) from 3D sMRI and patient's age.
  arXiv  Detail & Related papers  (2024-07-19T12:58:18Z)
• PheME: A deep ensemble framework for improving phenotype prediction from multi-modal data [42.56953523499849]
  We present PheME, an Ensemble framework using Multi-modality data of structured EHRs and unstructured clinical notes for accurate Phenotype prediction. We leverage ensemble learning to combine outputs from single-modal models and multi-modal models to improve phenotype predictions.
  arXiv  Detail & Related papers  (2023-03-19T23:41:04Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.