Computational Imaging Meets LLMs: Zero-Shot IDH Mutation Prediction in Brain Gliomas

• URL: http://arxiv.org/abs/2511.03376v1
• Date: Wed, 05 Nov 2025 11:31:08 GMT
• Title: Computational Imaging Meets LLMs: Zero-Shot IDH Mutation Prediction in Brain Gliomas
• Authors: Syed Muqeem Mahmood, Hassan Mohy-ud-Din,
• Abstract summary: We present a framework that combines Large Language Models with computational image analytics for non-invasive, zero-shot prediction of mutation status in brain gliomas.<n>We evaluated this framework on six publicly available datasets (N = 1427) and results showcased high accuracy and balanced classification performance.
• Score: 0.34983827101872134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a framework that combines Large Language Models with computational image analytics for non-invasive, zero-shot prediction of IDH mutation status in brain gliomas. For each subject, coregistered multi-parametric MRI scans and multi-class tumor segmentation maps were processed to extract interpretable semantic (visual) attributes and quantitative features, serialized in a standardized JSON file, and used to query GPT 4o and GPT 5 without fine-tuning. We evaluated this framework on six publicly available datasets (N = 1427) and results showcased high accuracy and balanced classification performance across heterogeneous cohorts, even in the absence of manual annotations. GPT 5 outperformed GPT 4o in context-driven phenotype interpretation. Volumetric features emerged as the most important predictors, supplemented by subtype-specific imaging markers and clinical information. Our results demonstrate the potential of integrating LLM-based reasoning with computational image analytics for precise, non-invasive tumor genotyping, advancing diagnostic strategies in neuro-oncology. The code is available at https://github.com/ATPLab-LUMS/CIM-LLM.

Related papers

• DCMIL: A Progressive Representation Learning of Whole Slide Images for Cancer Prognosis Analysis [20.81711023855778]
  We propose an easy-to-hard progressive representation learning, termed dual-curriculum contrastive multi-instance learning (DCMIL)<n>Experiments on twelve cancer types (5,954 patients, 12.54 million tiles) demonstrate that DCMIL outperforms standard WSI-based prognostic models.<n>DCMIL identifies fine-grained prognosis-salient regions, provides robust instance uncertainty estimation, and captures morphological differences between normal and tumor tissues.
  arXiv  Detail & Related papers  (2025-10-16T08:03:54Z)
• Adapting HFMCA to Graph Data: Self-Supervised Learning for Generalizable fMRI Representations [57.054499278843856]
  Functional magnetic resonance imaging (fMRI) analysis faces significant challenges due to limited dataset sizes and domain variability between studies.<n>Traditional self-supervised learning methods inspired by computer vision often rely on positive and negative sample pairs.<n>We propose adapting a recently developed Hierarchical Functional Maximal Correlation Algorithm (HFMCA) to graph-structured fMRI data.
  arXiv  Detail & Related papers  (2025-10-05T12:35:01Z)
• Performance of GPT-5 in Brain Tumor MRI Reasoning [4.156123728258067]
  Large language models (LLMs) have enabled visual question answering (VQA) approaches that integrate image interpretation with natural language reasoning.<n>We evaluated GPT-4o, GPT-5-nano, GPT-5-mini, and GPT-5 on a curated brain tumor VQA benchmark.<n>Results showed that GPT-5-mini achieved the highest macro-average accuracy (44.19%), followed by GPT-5 (43.71%), GPT-4o (41.49%), and GPT-5-nano (35.85%)
  arXiv  Detail & Related papers  (2025-08-14T17:35:31Z)
• BRISC: Annotated Dataset for Brain Tumor Segmentation and Classification [0.6840587119863303]
  We introduce BRISC, a dataset designed for brain tumor segmentation and classification tasks, featuring high-resolution segmentation masks.<n>The dataset comprises 6,000 contrast-enhanced T1-weighted MRI scans, which were collated from multiple public datasets that lacked segmentation labels.<n>It includes three major tumor types, namely glioma, meningioma, and pituitary, as well as non-tumorous cases.
  arXiv  Detail & Related papers  (2025-06-17T08:56:05Z)
• MIL vs. Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning [52.231128973251124]
  We compare various strategies for predicting survival at the WSI and patient level.<n>The former treats each WSI as an independent sample, mimicking the strategy adopted in other works.<n>The latter comprises methods to either aggregate the predictions of the several WSIs or automatically identify the most relevant slide.
  arXiv  Detail & Related papers  (2025-03-29T11:14:02Z)
• GBT-SAM: Adapting a Foundational Deep Learning Model for Generalizable Brain Tumor Segmentation via Efficient Integration of Multi-Parametric MRI Data [5.7802171590699984]
  We present GBT-SAM, a parameter-efficient deep learning framework that adapts the Segment Anything Model to mp-MRI data.<n>Our model is trained by a two-step fine-tuning strategy that incorporates a depth-aware module to capture inter-slice correlations.<n>It achieves a Dice Score of 93.54 on the BraTS Adult Glioma dataset and demonstrates robust performance on Meningioma, Pediatric Glioma, and Sub-Saharan Glioma datasets.
  arXiv  Detail & Related papers  (2025-03-06T11:18:22Z)
• A Multi-Modal Deep Learning Framework for Pan-Cancer Prognosis [15.10417643788382]
  In this paper, a deep-learning based model, named UMPSNet, is proposed.<n>UMPSNet integrates four types of important meta data (demographic information, cancer type information, treatment protocols, and diagnosis results) into text templates, and then introduces a text encoder to extract textual features.<n>By incorporating the multi-modality of patient data and joint training, UMPSNet outperforms all SOTA approaches.
  arXiv  Detail & Related papers  (2025-01-13T02:29:42Z)
• MRGen: Segmentation Data Engine for Underrepresented MRI Modalities [59.61465292965639]
  Training medical image segmentation models for rare yet clinically important imaging modalities is challenging due to the scarcity of annotated data.<n>This paper investigates leveraging generative models to synthesize data, for training segmentation models for underrepresented modalities.<n>We present MRGen, a data engine for controllable medical image synthesis conditioned on text prompts and segmentation masks.
  arXiv  Detail & Related papers  (2024-12-04T16:34:22Z)
• An Early Investigation into the Utility of Multimodal Large Language Models in Medical Imaging [0.3029213689620348]
  We explore the potential of the Gemini (textitgemini-1.0-pro-vision-latest) and GPT-4V models for medical image analysis. Both Gemini AI and GPT-4V are first used to classify real versus synthetic images, followed by an interpretation and analysis of the input images. Our early investigation presented in this work provides insights into the potential of MLLMs to assist with the classification and interpretation of retinal fundoscopy and lung X-ray images.
  arXiv  Detail & Related papers  (2024-06-02T08:29:23Z)
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images [68.8204255655161]
  We propose a deep-learning framework to detect and classify areas of microscopic images with similar levels of gene expression. The data we analyze requires an unsupervised learning model for which we employ a type of Artificial Neural Network - Deep Learning Autoencoders.
  arXiv  Detail & Related papers  (2023-04-19T13:45:28Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.