UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation

• URL: http://arxiv.org/abs/2605.17140v2
• Date: Tue, 19 May 2026 19:26:19 GMT
• Title: UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation
• Abstract summary: Brain tumor diagnosis is largely dependent on Magnetic Resonance Imaging (MRI) evaluation.<n>Despite increasing demands in radiology, this expertise is difficult to scale, straining the current health systems.<n>Vision-Language Models (VLMs) provide an opportunity to reduce this burden through a semi-automated, interactive interpretation of complex brain MRIs.
• Score: 1.2006896500048554
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Brain tumor diagnosis is largely dependent on Magnetic Resonance Imaging (MRI) evaluation, which requires radiologists to synthesize thousands of images across multiple 3D sequences and longitudinal studies. This process requires advanced neuro-radiology training, poses substantial cognitive load, and is highly time-consuming. Despite increasing demands in radiology, this expertise is difficult to scale, straining the current health systems. Vision-Language Models (VLMs) provide an opportunity to reduce this burden through a semi-automated, interactive interpretation of complex brain MRIs. However, they are currently underutilized in neuro-oncology due to a lack of specialized benchmarks for evaluating them. We introduce a clinically relevant visual question answering (VQA) benchmark -- the UCSF-PDGM-VQA dataset -- consisting of 2,387 QA pairs from 473 glioma-related MRI studies in the public UCSF-PDGM dataset. We further establish a performance baseline for six state-of-the-art vision-language models (VLMs) and one large language model on this dataset. We find that current models are incapable of effectively processing multi-sequence, 3-dimensional MRI scans, thus resulting in a suppression of visual features and over-reliance on language priors, causing modality collapse. These findings underscore a critical deficiency in current model reliability and safety within clinical settings, necessitating the development of robust, domain-specific VLMs.

Related papers

• How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning? [49.14050864978079]
  We introduce the Time-Aware Multi-View MRI Benchmark, an evaluation framework unifying multi-view anatomical input, temporal reasoning, and structured localization guidance.<n>The benchmark comprises 3,920 expert-verified question-answer pairs derived from 890 patients across over 3,200 longitudinal MRI timepoints.
  arXiv  Detail & Related papers  (2026-08-13T14:35:20Z)
• Unified Multimodal Model for Brain MRI Imputation and Understanding [19.00815560145768]
  We propose a novel unified multimodal model, UniBrain, for brain magnetic resonance image (MRI) analysis.<n>During training, an interleaved and description-enriched data flow is constructed to train the model in an autoregressive manner.<n>A self-alignment strategy is introduced to leverage dense image embeddings to learn fine-grained anatomical features without requiring detailed image captions.
  arXiv  Detail & Related papers  (2026-06-15T09:51:00Z)
• Multimodal Deep Learning for Dynamic and Static Neuroimaging: Integrating MRI and fMRI for Alzheimer Disease Analysis [5.338598022966635]
  We present a framework that integrates MRI and fMRI for classification of Alzheimer Disease (AD), Mild Cognitive Impairment, and Normal Cognitive State.<n> structural features are extracted from MRI using 3D convolutional neural networks, while temporal features are learned from fMRI sequences.<n>Results show that data augmentation substantially improves classification stability and generalization, particularly for the multimodal 3DCNN-LSTM model.
  arXiv  Detail & Related papers  (2026-03-09T12:58:45Z)
• Generative Latent Representations of 3D Brain MRI for Multi-Task Downstream Analysis in Down Syndrome [3.344873290507966]
  We develop variational autoencoders to encode 3D brain MRI scans into compact latent space representations for generative and predictive applications.<n>Our results demonstrate that the VAE successfully captures essential brain features while maintaining high reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-02-14T11:50:57Z)
• Decipher-MR: A Vision-Language Foundation Model for 3D MRI Representations [12.805804608410739]
  We present Decipher-MR, a 3D MRI-specific vision-language foundation model trained on a large-scale dataset.<n>Decipher-MR integrates self-supervised vision learning with report-guided text supervision to build robust, generalizable representations.<n>Our results establish Decipher-MR as a scalable and versatile foundation for MRI-based AI, facilitating efficient development across clinical and research domains.
  arXiv  Detail & Related papers  (2025-09-25T14:43:33Z)
• MRN: Harnessing 2D Vision Foundation Models for Diagnosing Parkinson's Disease with Limited 3D MR Data [0.6183104361749774]
  Current clinical practice often relies on diagnostic biomarkers in QSM and NM-MRI images.<n>We address these challenges by leveraging 2D vision foundation models (VFMs)<n>Our approach achieved first place in the MICCAI 2025 PDCADxFoundation challenge, with an accuracy of 86.4% trained on a dataset of only 300 labeled QSM and NM-MRI scans.
  arXiv  Detail & Related papers  (2025-09-22T10:59:27Z)
• Building a General SimCLR Self-Supervised Foundation Model Across Neurological Diseases to Advance 3D Brain MRI Diagnoses [2.4836875944302634]
  We present a general, high-resolution SimCLR-based SSL foundation model for 3D brain structural MRI.<n>Our model still achieves superior performance when fine-tuned using only 20% of labeled training samples for predicting Alzheimer's disease.
  arXiv  Detail & Related papers  (2025-09-12T18:05:08Z)
• Towards a general-purpose foundation model for fMRI analysis [58.06455456423138]
  We introduce NeuroSTORM, a framework that learns from 4D fMRI volumes and enables efficient knowledge transfer across diverse applications.<n>NeuroSTORM is pre-trained on 28.65 million fMRI frames (>9,000 hours) from over 50,000 subjects across multiple centers and ages 5 to 100.<n>It outperforms existing methods across five tasks: age/gender prediction, phenotype prediction, disease diagnosis, fMRI-to-image retrieval, and task-based fMRI.
  arXiv  Detail & Related papers  (2025-06-11T23:51:01Z)
• A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
  We propose a unified MRI reconstruction model robust to various measurement undersampling patterns and image resolutions.<n>Our model improves SSIM by 11% and PSNR by 4 dB over a state-of-the-art CNN (End-to-End VarNet) with 600$times$ faster inference than diffusion methods.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Brain3D: Generating 3D Objects from fMRI [78.46936519561298]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject.<n>We show that our model captures the distinct functionalities of each region of human vision system.<n>Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Video4MRI: An Empirical Study on Brain Magnetic Resonance Image Analytics with CNN-based Video Classification Frameworks [60.42012344842292]
  3D CNN-based models dominate the field of magnetic resonance image (MRI) analytics. In this paper, four datasets of Alzheimer's and Parkinson's disease recognition are utilized in experiments. In terms of efficiency, the video framework performs better than 3D-CNN models by 5% - 11% with 50% - 66% less trainable parameters.
  arXiv  Detail & Related papers  (2023-02-24T15:26:31Z)
• Data and Physics Driven Learning Models for Fast MRI -- Fundamentals and Methodologies from CNN, GAN to Attention and Transformers [72.047680167969]
  This article aims to introduce the deep learning based data driven techniques for fast MRI including convolutional neural network and generative adversarial network based methods. We will detail the research in coupling physics and data driven models for MRI acceleration. Finally, we will demonstrate through a few clinical applications, explain the importance of data harmonisation and explainable models for such fast MRI techniques in multicentre and multi-scanner studies.
  arXiv  Detail & Related papers  (2022-04-01T22:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.