SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering

• URL: http://arxiv.org/abs/2608.25630v1
• Date: Wed, 26 Aug 2026 10:58:45 GMT
• Title: SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering
• Abstract summary: We introduce BreMRIs-VQA, a clinically curated breast MRI benchmark with 1.19M QA pairs from 71.0K sequences and 12.9K patients, covering both free-text and multiple-choice questions.<n>We propose SeVeR, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding.<n>Experiments on BreMRIs-VQA and public benchmarks show that SeVeR improves both discriminative and generative performance while exposing substantially fewer visual tokens.
• Score: 20.249900521542195
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modalities provide diverse diagnostic cues but expose the decoder to many repeated anatomical regions. To investigate reasoning under multi-sequence visual redundancy, we first introduce BreMRIs-VQA, a clinically curated breast MRI benchmark with 1.19M QA pairs from 71.0K sequences and 12.9K patients, covering both free-text and multiple-choice questions. We further propose SeVeR, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding, trained with a marginal-utility self-consistency objective that suppresses unhelpful retrieval. Experiments on BreMRIs-VQA and public benchmarks show that SeVeR improves both discriminative and generative performance while exposing substantially fewer visual tokens.

Related papers

• MedFG-VQA: Low-Frequency Memory and Graph Attention for Lightweight Medical VQA [55.12170695219291]
  MedFG-VQA is a lightweight framework that augments DCT-based low-frequency features and employs graph-enhanced cross-attention for effective visual-textual alignment.<n>To address data scarcity, we construct SynMed-VQA, a large-scale synthetic dataset comprising over 2 million question-answer pairs across 9 imaging modalities and 10 major organs.
  arXiv  Detail & Related papers  (2026-08-27T09:16:14Z)
• Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence [51.57735743553791]
  We introduce FundusGround, a new benchmark for clinically interpretable ophthalmic VQA with spatially-grounded lesion evidence.<n>Built upon this structured lesion evidence, 72,706 questions are then generated spanning four formats.<n>Experiments demonstrate that incorporating lesion-level visual evidence consistently improves model performance and transparency.
  arXiv  Detail & Related papers  (2026-05-21T12:37:03Z)
• UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation [1.2006896500048554]
  Brain tumor diagnosis is largely dependent on Magnetic Resonance Imaging (MRI) evaluation.<n>Despite increasing demands in radiology, this expertise is difficult to scale, straining the current health systems.<n>Vision-Language Models (VLMs) provide an opportunity to reduce this burden through a semi-automated, interactive interpretation of complex brain MRIs.
  arXiv  Detail & Related papers  (2026-05-16T20:10:19Z)
• Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRI [22.439797107933355]
  We introduceSGMRI-VQA, a benchmark for multi-frame, spatially grounded reasoning on volumetric MRI.<n>Each QA pair includes a clinician-aligned chain-of-thought trace with frame-indexed bounding box coordinates.<n>We show that supervised fine-tuning of Qwen3-VL-8B with bounding box supervision consistently improves grounding performance over strong zero-shot baselines.
  arXiv  Detail & Related papers  (2026-04-17T08:06:39Z)
• brat: Aligned Multi-View Embeddings for Brain MRI Analysis [36.795218160666266]
  brat is a multi-view representation learning framework for brain magnetic resonance imaging (MRI) trained on MRIs paired with clinical reports.<n>Brain MRIs present unique challenges due to the presence of numerous, highly varied, and often subtle abnormalities that are localized to a few slices within a 3D volume.
  arXiv  Detail & Related papers  (2025-12-21T10:37:31Z)
• TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models [54.48710348910535]
  Existing medical reasoning benchmarks primarily focus on analyzing a patient's condition based on an image from a single visit.<n>We introduce TemMed-Bench, the first benchmark designed for analyzing changes in patients' conditions between different clinical visits.
  arXiv  Detail & Related papers  (2025-09-29T17:51:26Z)
• Are Vision Language Models Ready for Clinical Diagnosis? A 3D Medical Benchmark for Tumor-centric Visual Question Answering [8.185551155349241]
  Vision-Language Models (VLMs) have shown promise in various 2D visual tasks, yet their readiness for 3D clinical diagnosis remains unclear.<n>We present DeepTumorVQA, a diagnostic visual question answering benchmark targeting abdominal tumors in CT scans.<n>It comprises 9,262 CT volumes (3.7M slices) from 17 public datasets, with 395K expert-level questions spanning four categories: Recognition, Measurement, Visual Reasoning, and Medical Reasoning.
  arXiv  Detail & Related papers  (2025-05-25T00:50:15Z)
• Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRI [16.725225424047256]
  We propose a sequence-to-sequence generation framework (Seq2Seq) for imaging-differentiation representation learning. In this study, not only do we propose arbitrary 3D/4D sequence generation within one model to generate any specified target sequence, but also we are able to rank the importance of each sequence. We conduct extensive experiments using three datasets including a toy dataset of 20,000 simulated subjects, a brain MRI dataset of 1,251 subjects, and a breast MRI dataset of 2,101 subjects.
  arXiv  Detail & Related papers  (2023-02-01T15:37:35Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)
• Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution [56.622832383316215]
  We propose a novel multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple input tensors. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention.
  arXiv  Detail & Related papers  (2022-04-08T07:56:55Z)
• Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation [75.58395328700821]
  We propose a novel model, Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the subjects/patients and sub-modalities correlations. We show the applicability of MGP-VAE on brain tumor segmentation where either, two, or three of four sub-modalities may be missing.
  arXiv  Detail & Related papers  (2021-07-07T19:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.