MC-CXR: A Multi-Context Chest X-ray Benchmark for Context-Induced Disruption in Vision-Language Models

• URL: http://arxiv.org/abs/2608.24118v1
• Date: Tue, 25 Aug 2026 06:28:17 GMT
• Title: MC-CXR: A Multi-Context Chest X-ray Benchmark for Context-Induced Disruption in Vision-Language Models
• Abstract summary: Vision-language models (VLMs) are increasingly used in clinical pipelines where a chest X-ray is interpreted alongside retrieved reports, preliminary notes, or prior imaging.<n>Existing benchmarks measure whether models answer correctly in isolation, but not whether they preserve a correct image-only decision when plausible context conflicts with the image.<n>We introduce Multi-Context Chest X-ray (MC-CXR), a benchmark of 240 cases expanded into 2,522 instances that isolates context-induced disruption through paired perturbation.
• Score: 2.2090506971647144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) are increasingly used in clinical pipelines where a chest X-ray is interpreted alongside retrieved reports, preliminary notes, or prior imaging. Existing benchmarks measure whether models answer correctly in isolation, but not whether they preserve a correct image-only decision when plausible context conflicts with the image. We introduce Multi-Context Chest X-ray (MC-CXR), a benchmark of 240 cases expanded into 2,522 instances that isolates context-induced disruption through paired perturbation. Each case fixes the current image and target finding while presenting matched reliable and misleading context across text and prior CXR, with visual overlays where available. MC-CXR defines three task families and two paired metrics, the switch-to-wrong rate and the context-aligned error rate. We evaluate ten VLMs spanning open-source general, medical-domain, and closed-source systems. Image-only accuracy is necessary but insufficient. Mean switch rates range from 45.6-78.1% across misleading textual sources and 35.7-61.7% across misleading visual sources. Among switched predictions, 74.6% align with the misleading label for text versus 17.6% for visual context, a 57.0-point gap (95% CI 50.9-62.8). This text-visual asymmetry is observed under the standardized direct-answer protocol. The dataset is available on PhysioNet.

Related papers

• When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning [73.56343820592399]
  We introduce Visualized Task Semantics (VTS), a controlled intervention that moves the question into the image while keeping the source problem and answer fixed.<n>Across six MLLMs and four benchmarks, accuracy drops in all 24 model-task pairs, by 17.8 points on average.<n>We present prompt-region grounding, whose core design aligns the question region with typed semantics and recovers its clean representation from a masked view.
  arXiv  Detail & Related papers  (2026-08-05T11:46:15Z)
• Beyond Static Anchors: Bounded Prototype Conditioning for Language-Free Medical Anomaly Detection [52.986874008247554]
  We propose ReCAP, a language-free framework that replaces static anchors with input-conditioned visual prototypes.<n>ReCAP re-centers separated normal and abnormal prototypes for each image through a bounded gated modulation.<n>It achieves the best image-level AUROC on all zero-shot and 23 of 24 few-shot settings, and the best zero-shot pixel-level AUROC on all three segmentation datasets.
  arXiv  Detail & Related papers  (2026-08-01T04:46:55Z)
• CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography [8.663183634036624]
  Vision-language models offer a natural interface for text-to-image retrieval.<n>Current biomedical models are primarily optimized for report-to-image matching.<n>We introduce CXR-Retrieve, a structured benchmark for compositional chest X-ray text-to-image retrieval.
  arXiv  Detail & Related papers  (2026-07-30T07:10:59Z)
• Vision-language models for chest radiography do not always need the image [4.363746910191589]
  Medical vision-language models report strong chest radiograph accuracy, and this is increasingly read as evidence that they use the image.<n>We introduce a causal audit that intervenes on the image, occluding the relevant region, occluding an irrelevant one, and swapping in another patient's same-label scan.<n>Across nine systems, a text-only model with no image access reaches within 5.7 accuracy points of the best multimodal one.
  arXiv  Detail & Related papers  (2026-06-16T09:22:10Z)
• Cross-modal linkage risk in clinical vision-language models [1.828619151598757]
  Vision-language models (VLMs) trained on paired chest radiographs and radiology reports learn a shared embedding space that can preserve instance-level image-report correspondence.<n>This poses a privacy risk in settings where radiographs and reports are deliberately kept separate after acquisition.<n>We formalized this as image-to-report retrieval and used public paired cohorts to audit the risk rather than as the privacy scenario.
  arXiv  Detail & Related papers  (2026-06-01T14:01:46Z)
• Medical thinking with multiple images [44.04557445622649]
  We introduce MedThinkVQA, an expert-annotated benchmark for thinking with multiple images.<n>The dataset contains 8,067 cases, including 720 test cases, with an average of 6.62 images per case.<n>On the test set, the best closed-source models, Claude-4.6-Opus, Gemini-3-Pro, and GPT-5.2-xhigh, reach only 57.2%, 55.3%, and 54.9% accuracy.
  arXiv  Detail & Related papers  (2026-04-14T18:51:07Z)
• Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations [19.488236277427358]
  Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process.<n>We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications.
  arXiv  Detail & Related papers  (2025-10-13T09:28:22Z)
• Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios [49.53589774730807]
  Multimodal large language models (MLLMs) have recently achieved state-of-the-art performance on tasks ranging from visual question answering to video understanding.<n>We reveal a response uncertainty phenomenon: twelve state-of-the-art open-source MLLMs overturn a previously correct answer in 65% of cases after receiving a single deceptive cue.
  arXiv  Detail & Related papers  (2024-11-05T01:11:28Z)
• Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation [91.63262242041695]
  We propose a novel Adaptive patch-word Matching (AdaMatch) model to correlate chest X-ray (CXR) image regions with words in medical reports. AdaMatch exploits the fine-grained relation between adaptive patches and words to provide explanations of specific image regions with corresponding words. In order to provide explicit explainability for CXR-report generation task, we propose an AdaMatch-based bidirectional large language model for Cyclic CXR-report generation.
  arXiv  Detail & Related papers  (2023-12-13T11:47:28Z)
• Improving Vision Anomaly Detection with the Guidance of Language Modality [64.53005837237754]
  This paper tackles the challenges for vision modality from a multimodal point of view. We propose Cross-modal Guidance (CMG) to tackle the redundant information issue and sparse space issue. To learn a more compact latent space for the vision anomaly detector, CMLE learns a correlation structure matrix from the language modality.
  arXiv  Detail & Related papers  (2023-10-04T13:44:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.