NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives

• URL: http://arxiv.org/abs/2608.03187v1
• Date: Tue, 04 Aug 2026 06:26:50 GMT
• Title: NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives
• Abstract summary: We present NeuroMosaic, a 3D multimodal language model that converts multi-sequence brain MRI into anatomy-indexed regional tokens.<n>The architecture combines a multi-resolution volumetric tokenizer, a neuroanatomical graph router, a molecular concept memory, and selective risk control.
• Score: 0.7742420215649948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal medical large language models remain structurally weak for neuro-oncology because volumetric evidence is compressed into generic visual tokens and diagnostic conclusions often lack an auditable link to MRI regions. We present NeuroMosaic, a 3D multimodal language model that converts multi-sequence brain MRI into anatomy-indexed regional tokens, aligns them with clinical narrative and molecular concepts, and generates evidence-linked outputs. The architecture combines a multi-resolution volumetric tokenizer, a neuroanatomical graph router, a molecular concept memory, and selective risk control. Across four glioma cohorts, NeuroMosaic achieved an internal subtype macro-F1 of 0.827 and external macro-F1 values of 0.784, 0.761, and 0.742. On UPenn-GBM, it improved over the strongest matched-input baseline by 3.6 percentage points (95% CI: 1.8 to 5.4, adjusted p = 0.0018), with IDH, 1p/19q, and MGMT AUROCs of 0.918, 0.861, and 0.781. Evidence pointing accuracy reached 0.703, and targeted evidence deletion reduced correct-answer probability by 0.187, compared with 0.046 for random deletion. These results establish anatomy-indexed routing as a measurable mechanism for accurate, grounded, and calibrated volumetric medical-language reasoning.

Related papers

• CoMNeT: A MedNeXt-CorrDiff Framework for Volumetric Brain Tumor Segmentation [1.2048008992808203]
  We show that combining a modern 3D convolutional segmentation model with corrective diffusion-based refinement and ensembling improves volumetric glioma segmentation.<n>We propose CoMNeT, a MedNeXt-CorrDiff framework that uses four MRI modalities as input and predicts ET, TC, and whole tumor.<n>Using five-fold cross-validation, CoMNeT achieved the highest Dice score for most tumor regions, with ET, TC, WT, and average Dice scores of 0.7543 +/- 0.0261, 0.6806 +/- 0.0166, 0.9049 +/- 0.0128, and 0.7798 +/- 0.0184
  arXiv  Detail & Related papers  (2026-06-13T13:51:10Z)
• Artemis: Anatomy-Resolved inTervention for Eliminating Multimodal NeuroImage confounderS [39.37223141220675]
  Multimodal neuroimaging enables non-invasive analysis of brain networks using graph neural networks.<n> demographic factors such as age and sex systematically confound the relationship between brain connectivity and clinical outcomes.<n>We propose Artemis, a region-level causal framework that bridges this gap with causal intervention at each brain region independently.
  arXiv  Detail & Related papers  (2026-06-10T05:10:36Z)
• Linking spatial biology and clinical histology via Haiku [35.723756132708026]
  We present Haiku, a tri-modal contrastive learning model trained on multiplexed immunofluorescence (mIF)<n>It comprises 26.7 million spatial patches from 3,218 tissue sections across 1,606 patients spanning 11 organ types.
  arXiv  Detail & Related papers  (2026-04-30T18:53:32Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• NeuroRAD-FM: A Foundation Model for Neuro-Oncology with Distributionally Robust Training [12.168374910250572]
  Existing foundation models (FMs) perform poorly in predicting uncommon molecular markers.<n>We developed a neuro-oncology specific FM with a distributionally robust loss function, enabling accurate estimation of tumor phenotypes.<n>For survival, c-index improved at all sites: CUIMC 0.592 to 0.597, UPenn 0.647 to 0.672, UCSF 0.600 to 0.627.
  arXiv  Detail & Related papers  (2025-09-18T20:43:08Z)
• Fusion-Based Brain Tumor Classification Using Deep Learning and Explainable AI, and Rule-Based Reasoning [0.0]
  This study presents an ensemble-based deep learning framework that combines MobileNetV2 and DenseNet121 convolutional neural networks (CNNs)<n>The models were trained and evaluated on the Figshare dataset using a stratified 5-fold cross-validation protocol.<n>The ensemble achieved superior performance compared to individual CNNs, with an accuracy of 91.7%, precision of 91.9%, recall of 91.7%, and F1-score of 91.6%.
  arXiv  Detail & Related papers  (2025-08-09T08:46:36Z)
• Explainable Anatomy-Guided AI for Prostate MRI: Foundation Models and In Silico Clinical Trials for Virtual Biopsy-based Risk Assessment [3.5408411348831232]
  We present a fully automated, anatomically guided deep learning pipeline for prostate cancer (PCa) risk stratification using routine MRI.<n>The pipeline integrates three key components: an nnU-Net module for segmenting the prostate gland and its zones on axial T2-weighted MRI; a classification module based on the DiceedPT Swin Transformer foundation model, fine-tuned on 3D patches with optional anatomical priors and clinical data; and a VAE-GAN framework for generating counterfactual heatmaps that localize decision-driving image regions.
  arXiv  Detail & Related papers  (2025-05-23T14:40:09Z)
• Predicting Length of Stay in Neurological ICU Patients Using Classical Machine Learning and Neural Network Models: A Benchmark Study on MIMIC-IV [49.1574468325115]
  This study explores multiple ML approaches for predicting LOS in ICU specifically for the patients with neurological diseases based on the MIMIC-IV dataset.<n>The evaluated models include classic ML algorithms (K-Nearest Neighbors, Random Forest, XGBoost and CatBoost) and Neural Networks (LSTM, BERT and Temporal Fusion Transformer)
  arXiv  Detail & Related papers  (2025-05-23T14:06:42Z)
• Full-Head Segmentation of MRI with Abnormal Brain Anatomy: Model and Data Release [1.9492386975158917]
  We collected 98 MRIs with volumetric segmentation labels for a diverse set of human subjects.<n>We developed a MultiAxial network consisting of three 2D U-Net that operate independently in sagittal, axial and coronal planes.
  arXiv  Detail & Related papers  (2025-01-30T19:31:13Z)
• AXIAL: Attention-based eXplainability for Interpretable Alzheimer's Localized Diagnosis using 2D CNNs on 3D MRI brain scans [43.06293430764841]
  This study presents an innovative method for Alzheimer's disease diagnosis using 3D MRI designed to enhance the explainability of model decisions. Our approach adopts a soft attention mechanism, enabling 2D CNNs to extract volumetric representations. With voxel-level precision, our method identified which specific areas are being paid attention to, identifying these predominant brain regions.
  arXiv  Detail & Related papers  (2024-07-02T16:44:00Z)
• Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation [42.06416052431378]
  2D radiology captioning is incompetent to reflect the real-world diagnostic challenge in the volumetric 3D anatomy. We collected an 18,885 text-scan pairs 3D-BrainCT dataset and applied clinical visual instruction tuning to train BrainGPT models to generate radiology-adherent 3D brain CT reports. Our work embodies a holistic framework that showcased the first-hand experience of curating a 3D brain CT dataset, fine-tuning anatomy-sensible language models, and proposing robust radiology evaluation metrics.
  arXiv  Detail & Related papers  (2024-07-02T12:58:35Z)
• MRI-based classification of IDH mutation and 1p/19q codeletion status of gliomas using a 2.5D hybrid multi-task convolutional neural network [0.18374319565577152]
  Isocitrate dehydrogenase mutation and 1p/19q codeletion status are important prognostic markers for glioma. Our goal was to develop artificial intelligence-based methods to non-invasively determine these molecular alterations from MRI. A 2.5D hybrid convolutional neural network was proposed to simultaneously localize the tumor and classify its molecular status.
  arXiv  Detail & Related papers  (2022-10-07T18:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.