CheXpercept: A Benchmark for Evaluating Expert-Level Lesion Perception in Chest X-rays

• URL: http://arxiv.org/abs/2606.21020v1
• Date: Fri, 19 Jun 2026 01:10:24 GMT
• Title: CheXpercept: A Benchmark for Evaluating Expert-Level Lesion Perception in Chest X-rays
• Abstract summary: CheXpercept is a sequential, multi-level perception benchmark for chest X-ray analysis.<n>It mirrors a radiologist's cognitive workflow across coarse-level detection, fine-level contour evaluation and revision, and semantic-level attribute extraction.<n>CheXpercept contains 10,400 QA items derived from 2,100 CXRs, covering seven clinically critical pulmonary and cardiac lesions.
• Score: 13.566972599173084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The evaluation of vision-language models (VLMs) for chest X-ray (CXR) analysis has largely been limited to disease-presence classification without visual grounding. Such evaluations fail to verify the expert-level lesion perception necessary to ensure the clinical reliability of VLMs. To address these limitations, we introduce CheXpercept, a sequential, multi-level perception benchmark that mirrors a radiologist's cognitive workflow across coarse-level detection, fine-level contour evaluation and revision, and semantic-level attribute extraction. To ensure high clinical fidelity at scale, we construct the dataset using a semi-automated generation pipeline paired with a review by six medical experts. CheXpercept contains 10,400 QA items derived from 2,100 CXRs, covering seven clinically critical pulmonary and cardiac lesions. To demonstrate the current landscape of VLM perception, we benchmark 14 general and medical VLMs on CheXpercept. The models achieve adequate performance only at the coarse level, with accuracy degrading precipitously on deeper visual tasks. Notably, medical VLMs show almost no perceptual advantage over their general-domain counterparts, highlighting a systemic flaw in current domain adaptation. The code and dataset will be publicly available.

Related papers

• SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation [1.5782980044380892]
  Current evaluation protocols for Vision-Language Models in Radiology Report Generation rely on report-level metrics that measure lexical overlap or aggregate clinical correctness.<n>We introduce SHOVIR, a benchmark for evaluating vision shortcut behavior in RRG.
  arXiv  Detail & Related papers  (2026-06-29T12:17:35Z)
• Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs [63.535652574541764]
  Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis.<n>Their application in gastrointestinal endoscopy is currently hindered by two critical limitations.<n>We propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-21T07:47:37Z)
• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data [76.89269238957593]
  Coronary artery stenosis is a leading cause of cardiovascular disease, diagnosed by analyzing the coronary arteries from multiple angiography views.<n>We propose SegmentMIL, a transformer-based multi-view multiple-instance learning framework for patient-level stenosis classification.
  arXiv  Detail & Related papers  (2026-02-02T13:07:52Z)
• Measuring and Aligning Abstraction in Vision-Language Models with Medical Taxonomies [9.399546516030757]
  Vision-Language Models show strong zero-shot performance for chest X-ray classification, but standard flat metrics fail to distinguish between clinically minor and severe errors.<n>We benchmark several state-of-the-art VLMs using hierarchical metrics and introduce Catastrophic Abstraction Errors to capture cross-branch mistakes.
  arXiv  Detail & Related papers  (2026-01-21T09:58:50Z)
• A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice [83.11942224668127]
  Janus-Pro-CXR (1B) is a chest X-ray interpretation system based on DeepSeek Janus-Pro model.<n>Our system outperforms state-of-the-art X-ray report generation models in automated report generation.
  arXiv  Detail & Related papers  (2025-12-23T13:26:13Z)
• Adaptive Detector-Verifier Framework for Zero-Shot Polyp Detection in Open-World Settings [7.768426131383283]
  Existing approaches struggle with the domain gap between controlled laboratory conditions and clinical practice.<n>We propose AdaptiveDetector, a novel two-stage detector-verifier framework comprising a YOLOv11 detector with a vision-language model (VLM) verifier.<n>This combination of adaptive thresholding and cost-sensitive reinforcement learning achieves clinically aligned, open-world polyp detection with substantially fewer false negatives.
  arXiv  Detail & Related papers  (2025-12-13T23:33:05Z)
• XBench: A Comprehensive Benchmark for Visual-Language Explanations in Chest Radiography [6.447908430647854]
  We present the first systematic benchmark for evaluating cross-modal interpretability in chest X-rays.<n>We generate visual explanations using cross-attention and similarity-based localization maps.<n>We quantitatively assess their alignment with radiologist-annotated regions across multiple pathologies.
  arXiv  Detail & Related papers  (2025-10-22T13:52:19Z)
• EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis [62.00431604976949]
  EndoBench is the first comprehensive benchmark specifically designed to assess MLLMs across the full spectrum of endoscopic practice.<n>We benchmark 23 state-of-the-art models, including general-purpose, medical-specialized, and proprietary MLLMs.<n>Our experiments reveal: proprietary MLLMs outperform open-source and medical-specialized models overall, but still trail human experts.
  arXiv  Detail & Related papers  (2025-05-29T16:14:34Z)
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references.<n>We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey.<n>Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)
• Chest X-ray Foundation Model with Global and Local Representations Integration [13.736829173377355]
  CheXFound is a vision foundation model that learns robust CXR representations and generalizes effectively across a wide range of downstream tasks.<n>We pretrain CheXFound on a curated CXR-1M dataset, comprising over one million unique CXRs from publicly available sources.<n>Our experimental results show that CheXFound outperforms state-of-the-art models in classifying 40 disease findings across different prevalence levels.
  arXiv  Detail & Related papers  (2025-02-07T18:16:15Z)
• A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation [22.8169684575764]
  Over 1.4 billion chest X-rays (CXRs) are performed annually due to their cost-effectiveness as an initial diagnostic test.<n>This scale of radiological studies provides a significant opportunity to streamline CXR interpretation and documentation.<n>We constructed a large-scale dataset (CheXinstruct), which we utilized to train a vision-language foundation model (CheXagent)
  arXiv  Detail & Related papers  (2024-01-22T18:51:07Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.