Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs

• URL: http://arxiv.org/abs/2508.14817v1
• Date: Wed, 20 Aug 2025 16:09:37 GMT
• Title: Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
• Authors: Skatje Myers, Dmitriy Dligach, Timothy A. Miller, Samantha Barr, Yanjun Gao, Matthew Churpek, Anoop Mayampurath, Majid Afshar,
• Abstract summary: Large language models (LLMs) offer a promising solution for extracting and reasoning over unstructured text.<n>Retrieval-augmented generation (RAG) offers an alternative by retrieving task-relevant passages from across the entire EHR.<n>Our results suggest that RAG remains a competitive and efficient approach even as newer models become capable of handling increasingly longer amounts of text.
• Score: 7.692452997544737
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Electronic health records (EHRs) are long, noisy, and often redundant, posing a major challenge for the clinicians who must navigate them. Large language models (LLMs) offer a promising solution for extracting and reasoning over this unstructured text, but the length of clinical notes often exceeds even state-of-the-art models' extended context windows. Retrieval-augmented generation (RAG) offers an alternative by retrieving task-relevant passages from across the entire EHR, potentially reducing the amount of required input tokens. In this work, we propose three clinical tasks designed to be replicable across health systems with minimal effort: 1) extracting imaging procedures, 2) generating timelines of antibiotic use, and 3) identifying key diagnoses. Using EHRs from actual hospitalized patients, we test three state-of-the-art LLMs with varying amounts of provided context, using either targeted text retrieval or the most recent clinical notes. We find that RAG closely matches or exceeds the performance of using recent notes, and approaches the performance of using the models' full context while requiring drastically fewer input tokens. Our results suggest that RAG remains a competitive and efficient approach even as newer models become capable of handling increasingly longer amounts of text.

Related papers

• Dual-stage and Lightweight Patient Chart Summarization for Emergency Physicians [31.476936654094942]
  We present a two-stage summarization system that runs entirely on embedded devices.<n>The retrieval stage uses locally stored EHRs, splits long notes into semantically coherent sections, and searches for the most relevant sections per query.<n>The generation stage uses a locally hosted small language model (SLM) to produce the summary from the retrieved text.
  arXiv  Detail & Related papers  (2025-10-05T19:30:56Z)
• TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models [54.48710348910535]
  Existing medical reasoning benchmarks primarily focus on analyzing a patient's condition based on an image from a single visit.<n>We introduce TemMed-Bench, the first benchmark designed for analyzing changes in patients' conditions between different clinical visits.
  arXiv  Detail & Related papers  (2025-09-29T17:51:26Z)
• Applications of Small Language Models in Medical Imaging Classification with a Focus on Prompt Strategies [7.998802413749936]
  This study investigates the performance of small language models (SLMs) in a medical imaging classification task.<n>Using the NIH Chest X-ray dataset, we evaluate multiple SLMs on the task of classifying chest X-ray positions.<n>Our results show that certain SLMs achieve competitive accuracy with well-crafted prompts.
  arXiv  Detail & Related papers  (2025-08-18T21:48:45Z)
• HC-LLM: Historical-Constrained Large Language Models for Radiology Report Generation [89.3260120072177]
  We propose a novel Historical-Constrained Large Language Models (HC-LLM) framework for Radiology report generation.<n>Our approach extracts both time-shared and time-specific features from longitudinal chest X-rays and diagnostic reports to capture disease progression.<n> Notably, our approach performs well even without historical data during testing and can be easily adapted to other multimodal large models.
  arXiv  Detail & Related papers  (2024-12-15T06:04:16Z)
• RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models [35.60385437194243]
  Current Medical Large Vision Language Models (Med-LVLMs) frequently encounter factual issues. RAG, which utilizes external knowledge, can improve the factual accuracy of these models but introduces two major challenges. We propose RULE, which consists of two components. First, we introduce a provably effective strategy for controlling factuality risk through the selection of retrieved contexts. Second, based on samples where over-reliance on retrieved contexts led to errors, we curate a preference dataset to fine-tune the model.
  arXiv  Detail & Related papers  (2024-07-06T16:45:07Z)
• Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data [7.815738943706123]
  Large Language Models (LLMs) are traditionally tailored for natural language processing. This research investigates the adaptability of LLMs, like GPT-4, to EHR data. In response to the longitudinal, sparse, and knowledge-infused nature of EHR data, our prompting approach involves taking into account specific characteristics.
  arXiv  Detail & Related papers  (2024-01-25T20:14:50Z)
• Question-Answering Based Summarization of Electronic Health Records using Retrieval Augmented Generation [0.0]
  We propose a method that mitigates shortcomings by combining semantic search, retrieval augmented generation and question-answering. Our approach is quite efficient; requires minimal to no training; does not suffer from the 'hallucination' problem of LLMs. It ensures diversity, since the summary will not have repeated content but diverse answers to specific questions.
  arXiv  Detail & Related papers  (2024-01-03T00:09:34Z)
• Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
  Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning. They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
  arXiv  Detail & Related papers  (2023-05-30T22:05:11Z)
• Self-supervised Answer Retrieval on Clinical Notes [68.87777592015402]
  We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. We report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages.
  arXiv  Detail & Related papers  (2021-08-02T10:42:52Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)
• Learning Contextualized Document Representations for Healthcare Answer Retrieval [68.02029435111193]
  Contextual Discourse Vectors (CDV) is a distributed document representation for efficient answer retrieval from long documents. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking.
  arXiv  Detail & Related papers  (2020-02-03T15:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.