NeuroQA: A Large-Scale Image-Grounded Benchmark for 3D Brain MRI Understanding

• URL: http://arxiv.org/abs/2605.20525v1
• Date: Tue, 19 May 2026 21:54:12 GMT
• Title: NeuroQA: A Large-Scale Image-Grounded Benchmark for 3D Brain MRI Understanding
• Abstract summary: NeuroQA is a large-scale benchmark for visual question answering in 3D brain magnetic resonance imaging (MRI)<n>It spans ages 5-104 and five clinical domains: Alzheimer's, Parkinson's, tumors, white matter disease, and neurodevelopment.<n>NeuroQA evaluates 11 clinically grounded reasoning skills across Yes/No, multiple-choice, and open-ended formats.
• Score: 6.217658756255346
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present NeuroQA, a large-scale benchmark for visual question answering in 3D brain magnetic resonance imaging (MRI), with 56,953 QA pairs from 12,977 subjects across 12 datasets. It spans ages 5-104 and five clinical domains: Alzheimer's, Parkinson's, tumors, white matter disease, and neurodevelopment. Unlike prior medical Visual Question Answering (VQA) efforts that operate on 2D slices or rely on narrow diagnostic labels, NeuroQA pairs every item with a full 3D volume. It evaluates 11 clinically grounded reasoning skills across Yes/No, multiple-choice, and open-ended formats. Of the 203 templates, 131 are image-grounded (answerable from a 3-plane viewer) and 72 are image-informed (ground truth from quantitative volumetry or clinical instruments). To remove text-only shortcuts, we apply answer-distribution refinement, reducing closed-format text-only accuracy from $>$80% to 44.6%; image necessity is assessed separately through an image-grounding protocol released with the benchmark. A 38-rule deterministic pipeline and two rounds of expert review verify every QA pair against FreeSurfer measurements, metadata, or radiology report fields, with zero same-subject contradictions across templates. We conduct a clinician evaluation in which two clinicians independently assess 100 frozen test items on a three-plane viewer. On closed-format (Yes/No + multiple-choice) test-public items, the best zero-shot vision-language model and a supervised 3D CNN baseline reach 47.5% and 43.7% accuracy respectively, both below the 49.4% text-only majority-template floor. NeuroQA adopts a two-tier release with public QA pairs for open-access datasets and reproducible generation scripts for datasets restricted by data use agreements (DUAs), plus subject-level splits, a held-out private test set, and an online leaderboard.

Related papers

• MedQA-MM: Shortcuts Behind Medical Visual Reasoning [45.333881843709804]
  A benchmark score credits final answers, but not the route by which an item can be answered.<n>In medical multimodal multiple-choice questions (MCQs), this distinction matters because a correct answer can be supported by the intended image finding.<n>Here, a route is an observable input path that can support answer selection, not a claim about the model's hidden cognition.
  arXiv  Detail & Related papers  (2026-09-03T01:40:19Z)
• Vision-language models for chest radiography do not always need the image [4.363746910191589]
  Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.<n>We introduce a causal audit that intervenes on the image, occluding the relevant region, occluding an irrelevant one, and swapping in another patient's same-label scan.<n>Across nine systems, a text-only model with no image access reaches within 5.7 accuracy points of the best multimodal one.
  arXiv  Detail & Related papers  (2026-06-16T09:22:10Z)
• MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence [24.517280048376758]
  medvigil is a 300-case evaluation suite drawn from four public medical VQA sources.<n>Every gold answer, refusal option, candidate-answer set, paraphrase, false-premise trap, ROI box, and clinical risk tier is clinician-authored.<n>We report seven correctness-conditioned audit metrics that summarise into the medvigil Composite Score (MCS)<n>The independent radiologist scores MCS 83.3 at silent-failure rate 5.8%, leaving a 14.1-point composite headroom above the strongest audited model.
  arXiv  Detail & Related papers  (2026-05-08T15:55:30Z)
• Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations [19.488236277427358]
  Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process.<n>We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications.
  arXiv  Detail & Related papers  (2025-10-13T09:28:22Z)
• Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding [112.46150793476603]
  We introduce Hulu-Med, a transparent, generalist medical Vision-Language Model (VLM)<n>Hulu-Med is trained on a curated corpus of 16.7 million samples, spanning 12 major anatomical systems and 14 medical imaging modalities.<n>Hulu-Med surpasses existing open-source models on 27 of 30 benchmarks and outperforms proprietary systems such as GPT-4o on 16 benchmarks.
  arXiv  Detail & Related papers  (2025-10-09T17:06:42Z)
• Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis [6.464464511743737]
  Vision-language models (VLMs) have shown promise in 2D medical image analysis, but extending them to 3D remains challenging.<n>We present Med3DVLM, a 3D VLM designed to address these challenges through three key innovations.<n>We evaluate our model on the M3D dataset, which includes radiology reports and VQA data for 120,084 3D medical images.
  arXiv  Detail & Related papers  (2025-03-25T20:09:30Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [56.99710477905796]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarios [53.94122089629544]
  We introduce CT-GLIP (Grounded Language-Image Pretraining with CT scans), a novel method that constructs organ-level image-text pairs to enhance multimodal contrastive learning. Our method, trained on a multimodal CT dataset comprising 44,011 organ-level vision-text pairs from 17,702 patients across 104 organs, demonstrates it can identify organs and abnormalities in a zero-shot manner using natural languages.
  arXiv  Detail & Related papers  (2024-04-23T17:59:01Z)
• M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models [49.5030774873328]
  Previous research has primarily focused on 2D medical images, leaving 3D images under-explored, despite their richer spatial information. We present a large-scale 3D multi-modal medical dataset, M3D-Data, comprising 120K image-text pairs and 662K instruction-response pairs. We also introduce a new 3D multi-modal medical benchmark, M3D-Bench, which facilitates automatic evaluation across eight tasks.
  arXiv  Detail & Related papers  (2024-03-31T06:55:12Z)
• Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
  A common scenario in developing countries is to have the volume metadata lost due multiple reasons. It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
  arXiv  Detail & Related papers  (2020-10-20T04:30:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.