RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection

• URL: http://arxiv.org/abs/2505.14318v2
• Date: Mon, 02 Jun 2025 05:56:06 GMT
• Title: RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection
• Authors: Wenjun Hou, Yi Cheng, Kaishuai Xu, Heng Li, Yan Hu, Wenjie Li, Jiang Liu,
• Abstract summary: Radar is a framework for enhancing radiology report generation with supplementary knowledge injection.<n>It first extracts the model's acquired knowledge that aligns with expert image-based classification outputs.<n>It then retrieves relevant supplementary knowledge to further enrich this information.
• Score: 15.640187325451237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in various domains, including radiology report generation. Previous approaches have attempted to utilize multimodal LLMs for this task, enhancing their performance through the integration of domain-specific knowledge retrieval. However, these approaches often overlook the knowledge already embedded within the LLMs, leading to redundant information integration. To address this limitation, we propose Radar, a framework for enhancing radiology report generation with supplementary knowledge injection. Radar improves report generation by systematically leveraging both the internal knowledge of an LLM and externally retrieved information. Specifically, it first extracts the model's acquired knowledge that aligns with expert image-based classification outputs. It then retrieves relevant supplementary knowledge to further enrich this information. Finally, by aggregating both sources, Radar generates more accurate and informative radiology reports. Extensive experiments on MIMIC-CXR, CheXpert-Plus, and IU X-ray demonstrate that our model outperforms state-of-the-art LLMs in both language quality and clinical accuracy.

Related papers

• Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation [77.10390725623125]
  retrieval-augmented generation (RAG) is widely employed to expand their knowledge scope.<n>Since RAG has shown promise in knowledge-intensive tasks like open-domain question answering, its broader application to complex tasks and intelligent assistants has further advanced its utility.<n>We present a systematic investigation of the intrinsic mechanisms by which RAGs integrate internal (parametric) and external (retrieved) knowledge.
  arXiv  Detail & Related papers  (2025-05-17T13:13:13Z)
• VLM-KG: Multimodal Radiology Knowledge Graph Generation [3.6669792968901573]
  We propose a novel framework for knowledge graph generation in radiology.<n>Our approach outperforms previous methods and introduces the first multimodal solution for radiology knowledge graph generation.
  arXiv  Detail & Related papers  (2025-05-13T06:11:10Z)
• Leveraging LLMs for Multimodal Retrieval-Augmented Radiology Report Generation via Key Phrase Extraction [0.0]
  We propose a retrieval-augmented generation approach that leverages multimodal retrieval and large language models.<n>Our method uses LLMs to extract key phrases from radiology reports, effectively focusing on essential diagnostic information.<n>We evaluate our approach on MIMIC-CXR dataset, achieving state-of-the-art results on CheXbert metrics and competitive RadGraph F1 metric.
  arXiv  Detail & Related papers  (2025-04-10T03:14:01Z)
• KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models [39.831976458410864]
  This paper presents KARGEN, a Knowledge-enhanced Automated radiology Report GENeration framework based on Large Language Models. The framework integrates a knowledge graph to unlock chest disease-related knowledge within the LLM to enhance the clinical utility of generated reports. Our approach demonstrates promising results on the MIMIC-CXR and IU-Xray datasets.
  arXiv  Detail & Related papers  (2024-09-09T06:57:22Z)
• MGH Radiology Llama: A Llama 3 70B Model for Radiology [50.42811030970618]
  This paper presents an advanced radiology-focused large language model: MGH Radiology Llama.<n>It is developed using the Llama 3 70B model, building upon previous domain-specific models like Radiology-GPT and Radiology-Llama2.<n>Our evaluation, incorporating both traditional metrics and a GPT-4-based assessment, highlights the enhanced performance of this work over general-purpose LLMs.
  arXiv  Detail & Related papers  (2024-08-13T01:30:03Z)
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
  Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation. Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge. RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
  arXiv  Detail & Related papers  (2024-05-10T02:48:45Z)
• Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning [16.849933628738277]
  Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists. This paper introduces a novel RRG method, textbfLM-RRG, that integrates large models (LMs) with clinical quality reinforcement learning. Experiments on the MIMIC-CXR and IU-Xray datasets demonstrate the superiority of our method over the state of the art.
  arXiv  Detail & Related papers  (2024-03-11T13:47:11Z)
• ExaRanker-Open: Synthetic Explanation for IR using Open-Source LLMs [60.81649785463651]
  We introduce ExaRanker-Open, where we adapt and explore the use of open-source language models to generate explanations. Our findings reveal that incorporating explanations consistently enhances neural rankers, with benefits escalating as the LLM size increases.
  arXiv  Detail & Related papers  (2024-02-09T11:23:14Z)
• LLM-driven Multimodal Target Volume Contouring in Radiation Oncology [46.23891509553877]
  Large language models (LLMs) can facilitate the integration of the textural information and images. We present a novel LLM-driven multimodal AI, namely LLMSeg, that is applicable to the challenging task of target volume contouring for radiation therapy. We demonstrate that the proposed model exhibits markedly improved performance compared to conventional unimodal AI models.
  arXiv  Detail & Related papers  (2023-11-03T13:38:42Z)
• Radiology-Llama2: Best-in-Class Large Language Model for Radiology [71.27700230067168]
  This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning. Quantitative evaluations using ROUGE metrics on the MIMIC-CXR and OpenI datasets demonstrate that Radiology-Llama2 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-08-29T17:44:28Z)
• Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment [27.111857943935725]
  We present an automatic, multi-modal approach for report generation from chest x-ray. Our approach features two distinct modules: (i) Learned knowledge base and (ii) Multi-modal alignment. With the aid of both modules, our approach clearly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-30T10:43:56Z)
• Knowledge Matters: Radiology Report Generation with General and Specific Knowledge [24.995748604459013]
  We propose a knowledge-enhanced radiology report generation approach. By merging the visual features of the radiology image with general knowledge and specific knowledge, the proposed model can improve the quality of generated reports.
  arXiv  Detail & Related papers  (2021-12-30T10:36:04Z)
• Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation [55.00308939833555]
  The PPKED includes three modules: Posterior Knowledge Explorer (PoKE), Prior Knowledge Explorer (PrKE) and Multi-domain Knowledge Distiller (MKD) PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias. PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias.
  arXiv  Detail & Related papers  (2021-06-13T11:10:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.