Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System

• URL: http://arxiv.org/abs/2412.06828v1
• Date: Fri, 06 Dec 2024 21:33:03 GMT
• Title: Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System
• Abstract summary: "RadCouncil" is a multi-agent Large Language Model (LLM) framework designed to enhance the generation of impressions in radiology reports from the finding section.<n>RadCouncil comprises three specialized agents: 1) a "Retrieval" Agent that identifies and retrieves similar reports from a vector database, 2) a "Radiologist" Agent that generates impressions based on the finding section of the given report plus the exemplar reports retrieved by the Retrieval Agent, and 3) a "Reviewer" Agent that evaluates the generated impressions and provides feedback.
• Score: 10.502391082887568
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study introduces "RadCouncil," a multi-agent Large Language Model (LLM) framework designed to enhance the generation of impressions in radiology reports from the finding section. RadCouncil comprises three specialized agents: 1) a "Retrieval" Agent that identifies and retrieves similar reports from a vector database, 2) a "Radiologist" Agent that generates impressions based on the finding section of the given report plus the exemplar reports retrieved by the Retrieval Agent, and 3) a "Reviewer" Agent that evaluates the generated impressions and provides feedback. The performance of RadCouncil was evaluated using both quantitative metrics (BLEU, ROUGE, BERTScore) and qualitative criteria assessed by GPT-4, using chest X-ray as a case study. Experiment results show improvements in RadCouncil over the single-agent approach across multiple dimensions, including diagnostic accuracy, stylistic concordance, and clarity. This study highlights the potential of utilizing multiple interacting LLM agents, each with a dedicated task, to enhance performance in specialized medical tasks and the development of more robust and adaptable healthcare AI solutions.

Related papers

• MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation [20.858060411945438]
  Automated 3D radiology report generation often suffers from clinical hallucinations and a lack of the iterative verification found in human practice.<n>We propose MARCH, a multi-agent framework that emulates the professional hierarchy of radiology departments and assigns specialized roles to distinct agents.<n>On the RadGenome-ChestCT dataset, MARCH significantly outperforms state-of-the-art baselines in both clinical fidelity and linguistic accuracy.
  arXiv  Detail & Related papers  (2026-04-17T15:42:03Z)
• Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation: Radiologist-Like Workflow with Clinically Verifiable Rewards [0.0]
  We propose MARL-Rad, a novel multi-modal multi-agent reinforcement learning framework for radiology report generation.<n>MarL-Rad coordinates region-specific agents and a global integrating agent, optimized via clinically verifiable rewards.
  arXiv  Detail & Related papers  (2026-02-17T12:48:32Z)
• Grounded by Experience: Generative Healthcare Prediction Augmented with Hierarchical Agentic Retrieval [29.377256313893934]
  Large language models (LLMs) offer a promising path to enhance healthcare predictions by drawing on their rich parametric knowledge.<n>LLMs are prone to factual inaccuracies due to limitations in the reliability and coverage of their embedded knowledge.<n>We propose GHAR, a underlinegenerative underlinehierarchical underlineagentic underlineRAG framework that simultaneously resolves when to retrieve and how to optimize the collaboration between submodules in healthcare.
  arXiv  Detail & Related papers  (2025-11-17T12:15:46Z)
• CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretation [62.0150409256153]
  We propose CXRAgent, a director-orchestrated, multi-stage agent for CXR interpretation.<n>The agent strategically orchestrates a set of CXR-analysis tools, with outputs normalized and verified by the Evidence-driven Validator.<n>Experiments on various CXR interpretation tasks show that CXRAgent delivers strong performance, providing visual evidence and generalizes well to clinical tasks of different complexity.
  arXiv  Detail & Related papers  (2025-10-24T10:31:30Z)
• Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation [0.2039123720459736]
  We introduce a multi-agent reinforcement learning framework that serves as a benchmark and evaluation environment for multimodal clinical reasoning in the radiology ecosystem.<n>The proposed framework integrates large language models (LLMs) and large vision models (LVMs) within a modular architecture composed of ten specialized agents responsible for image analysis, feature extraction, report generation, review, and evaluation.
  arXiv  Detail & Related papers  (2025-09-22T04:31:27Z)
• Comparative Analysis of Abstractive Summarization Models for Clinical Radiology Reports [3.065600760950715]
  This research explores the use of advanced abstractive summarization models to generate the concise impression from the findings section of a radiology report.<n>A comparative analysis is conducted on leading pre-trained and open-source large language models, including T5-base, BART-base, PEG-x-base, ChatGPT-4, LLaMA-3-8B, and a custom Pointer Generator Network with a coverage mechanism.
  arXiv  Detail & Related papers  (2025-06-19T12:07:17Z)
• MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research [70.72318131988102]
  MLR-Bench is a comprehensive benchmark for evaluating AI agents on open-ended machine learning research.<n>MLR-Bench includes three key components: (1) 201 research tasks sourced from NeurIPS, ICLR, and ICML workshops covering diverse ML topics; (2) MLR-Judge, an automated evaluation framework combining LLM-based reviewers with carefully designed review rubrics to assess research quality; and (3) MLR-Agent, a modular agent scaffold capable of completing research tasks through four stages: idea generation, proposal formulation, experimentation, and paper writing.
  arXiv  Detail & Related papers  (2025-05-26T13:18:37Z)
• A Multimodal Multi-Agent Framework for Radiology Report Generation [2.1477122604204433]
  Radiology report generation (RRG) aims to automatically produce diagnostic reports from medical images.<n>We propose a multimodal multi-agent framework for RRG that aligns with the stepwise clinical reasoning workflow.
  arXiv  Detail & Related papers  (2025-05-14T20:28:04Z)
• RadPhi-3: Small Language Models for Radiology [3.2515490312812623]
  In this work, we present RadPhi-3, a Small Language Model instruction tuned from Phi-3-mini-4k-instruct with 3.8B parameters to assist with various tasks in radiology. While impression summary generation has been the primary task which has been explored in prior works, we also explore other useful tasks like summary generation comparing the current radiology report and its prior report, section extraction from radiology reports, tagging the reports with various pathologies and tubes, lines or devices present in them etc.
  arXiv  Detail & Related papers  (2024-11-19T11:24:28Z)
• Resource-Efficient Medical Report Generation using Large Language Models [3.2627279988912194]
  Medical report generation is the task of automatically writing radiology reports for chest X-ray images. We propose a new framework leveraging vision-enabled Large Language Models (LLM) for the task of medical report generation.
  arXiv  Detail & Related papers  (2024-10-21T05:08:18Z)
• ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration [14.180987429395584]
  ColaCare is a framework that enhances Electronic Health Record (EHR) modeling through multi-agent collaboration driven by Large Language Models (LLMs) Our approach seamlessly integrates domain-specific expert models with LLMs to bridge the gap between structured EHR data and text-based reasoning.
  arXiv  Detail & Related papers  (2024-10-03T14:55:22Z)
• MGH Radiology Llama: A Llama 3 70B Model for Radiology [50.42811030970618]
  This paper presents an advanced radiology-focused large language model: MGH Radiology Llama. It is developed using the Llama 3 70B model, building upon previous domain-specific models like Radiology-GPT and Radiology-Llama2. Our evaluation, incorporating both traditional metrics and a GPT-4-based assessment, highlights the enhanced performance of this work over general-purpose LLMs.
  arXiv  Detail & Related papers  (2024-08-13T01:30:03Z)
• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data [115.0747462486285]
  ChatRadio-Valuer is a tailored model for automatic radiology report generation that learns generalizable representations. The clinical dataset utilized in this study encompasses a remarkable total of textbf332,673 observations. ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al.
  arXiv  Detail & Related papers  (2023-10-08T17:23:17Z)
• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data [66.9359934608229]
  This study aims to initiate the development of Radiology Foundation Model, termed as RadFM. To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans. We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
  arXiv  Detail & Related papers  (2023-08-04T17:00:38Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment [27.111857943935725]
  We present an automatic, multi-modal approach for report generation from chest x-ray. Our approach features two distinct modules: (i) Learned knowledge base and (ii) Multi-modal alignment. With the aid of both modules, our approach clearly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-30T10:43:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.