RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review

• URL: http://arxiv.org/abs/2603.06681v1
• Date: Wed, 04 Mar 2026 00:13:57 GMT
• Title: RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review
• Abstract summary: We present RADAR, a multimodal benchmark for radiology report discrepancy analysis.<n>It pairs 3D medical images with a preliminary report and corresponding candidate edits for the same study.<n> RADAR defines a structured discrepancy assessment task requiring models to evaluate proposed edits.
• Score: 10.986820518559341
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Radiology reports for the same patient examination may contain clinically meaningful discrepancies arising from interpretation differences, reporting variability, or evolving assessments. Systematic analysis of such discrepancies is important for quality assurance, clinical decision support, and multimodal model development, yet remains limited by the lack of standardized benchmarks. We present RADAR, a multimodal benchmark for radiology report discrepancy analysis that pairs 3D medical images with a preliminary report and corresponding candidate edits for the same study. The dataset reflects a standard clinical workflow in which trainee radiologists author preliminary reports that are subsequently reviewed and revised by attending radiologists. RADAR defines a structured discrepancy assessment task requiring models to evaluate proposed edits by determining image-level agreement, assessing clinical severity, and classifying edit type (correction, addition, or clarification). In contrast to prior work emphasizing binary error detection or comparison against fully independent reference reports, RADAR targets fine-grained clinical reasoning and image-text alignment at the report review stage. The benchmark consists of expert-annotated abdominal CT examinations and is accompanied by standardized evaluation protocols to support systematic comparison of multimodal models. RADAR provides a clinically grounded testbed for evaluating multimodal systems as reviewers of radiology report edits.

Related papers

• SPARC-Rad: A Multimodal Benchmark Dataset and Evaluation Pipeline for Spatial and Anatomical Reasoning in Radiology Vision-Language Models [10.383904797979445]
  Many available benchmarks emphasize disease classification, report generation, or broad visual question answering rather than the spatial and anatomical reasoning required for radiology.<n>We developed the Spatial Perception and Anatomical Reasoning in Clinical Radiology (SPARC-Rad) Benchmark, a manually curated multimodal benchmark dataset and evaluation pipeline.<n>Radiation trainees manually designed and annotated questions to evaluate anatomical identification, localization, laterality, regional recognition, device identification, and inter-structure spatial relationships.
  arXiv  Detail & Related papers  (2026-07-30T20:13:32Z)
• Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context [0.6070952062639758]
  Harrison.Rad 1.5 (HR1.5) is a radiology-specific multimodal large language model that accepts interleaved text and visual inputs.<n>It generates structured and unstructured text across plain-film radiology, spanning computed radiography, chest, musculoskeletal, abdominal, spine, and pelvic x-rays, and mammography.<n>HR1.5 is trained through a three-stage pipeline: domain adaptation of a base language model on radiology reports, contrastive vision-encoder training with curriculum-based hard negatives, and visual-question-answering fine-tuning on multi-turn conversations.
  arXiv  Detail & Related papers  (2026-07-07T06:23:08Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• A unified multi-task framework enables interpretable chest radiograph analysis [35.74517495754825]
  IMT-CXR is a framework that emulates radiologists' diagnostic workflow through three evidence-driven stages.<n>The framework employs a unified transformer architecture optimized via medical-domain instruction tuning.<n>Three radiologists rated 66% of AI-generated reports as comparable to or surpassing original clinical reports in diagnostic clarity.
  arXiv  Detail & Related papers  (2026-06-02T10:02:15Z)
• AgentsEval: Clinically Faithful Evaluation of Medical Imaging Reports via Multi-Agent Reasoning [73.50200033931148]
  We introduce AgentsEval, a multi-agent stream reasoning framework that emulates the collaborative diagnostic workflow of radiologists.<n>By dividing the evaluation process into interpretable steps including criteria definition, evidence extraction, alignment, and consistency scoring, AgentsEval provides explicit reasoning traces and structured clinical feedback.<n> Experimental results demonstrate that AgentsEval delivers clinically aligned, semantically faithful, and interpretable evaluations that remain robust under paraphrastic, semantic, and stylistic perturbations.
  arXiv  Detail & Related papers  (2026-01-23T11:59:13Z)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis [56.373297358647655]
  Retrieval-Augmented Diagnosis (RAD) is a novel framework that injects external knowledge into multimodal models directly on downstream tasks.<n>RAD operates through three key mechanisms: retrieval and refinement of disease-centered knowledge from multiple medical sources, a guideline-enhanced contrastive loss transformer, and a dual decoder.
  arXiv  Detail & Related papers  (2025-09-24T10:36:14Z)
• Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation [0.2039123720459736]
  We introduce a multi-agent reinforcement learning framework that serves as a benchmark and evaluation environment for multimodal clinical reasoning in the radiology ecosystem.<n>The proposed framework integrates large language models (LLMs) and large vision models (LVMs) within a modular architecture composed of ten specialized agents responsible for image analysis, feature extraction, report generation, review, and evaluation.
  arXiv  Detail & Related papers  (2025-09-22T04:31:27Z)
• Automated Structured Radiology Report Generation [11.965406008391371]
  We introduce Structured Radiology Report Generation (SRRG), a new task that reformulates free-text radiology reports into a standardized format.<n>We create a novel dataset by restructuring reports using large language models (LLMs) following strict structured reporting desiderata.<n>We also introduce SRR-BERT, a fine-grained disease classification model trained on 55 labels, enabling more precise and clinically informed evaluation of structured reports.
  arXiv  Detail & Related papers  (2025-05-30T05:23:01Z)
• Image-aware Evaluation of Generated Medical Reports [11.190146577567548]
  The paper proposes a novel evaluation metric for automatic medical report generation from X-ray images, VLScore. The key idea of our metric is to measure the similarity between radiology reports while considering the corresponding image. We demonstrate the benefit of our metric through evaluation on a dataset where radiologists marked errors in pairs of reports, showing notable alignment with radiologists' judgments.
  arXiv  Detail & Related papers  (2024-10-22T18:50:20Z)
• RaTEScore: A Metric for Radiology Report Generation [59.37561810438641]
  This paper introduces a novel, entity-aware metric, as Radiological Report (Text) Evaluation (RaTEScore) RaTEScore emphasizes crucial medical entities such as diagnostic outcomes and anatomical details, and is robust against complex medical synonyms and sensitive to negation expressions. Our evaluations demonstrate that RaTEScore aligns more closely with human preference than existing metrics, validated both on established public benchmarks and our newly proposed RaTE-Eval benchmark.
  arXiv  Detail & Related papers  (2024-06-24T17:49:28Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.