Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models

• URL: http://arxiv.org/abs/2508.06504v1
• Date: Fri, 25 Jul 2025 20:57:16 GMT
• Title: Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
• Abstract summary: We investigate the performance challenges of large language models (LLMs) for biomedical named entity recognition (NER)<n>In our approach, the annotated in-context learning examples are selected based on their similarities with the input texts, and the prompt is dynamically updated for each instance during inference.<n>We implemented and optimized static and dynamic prompt engineering techniques and evaluated them on five biomedical NER datasets.
• Score: 4.848166051841711
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biomedical named entity recognition (NER) is a high-utility natural language processing (NLP) task, and large language models (LLMs) show promise particularly in few-shot settings (i.e., limited training data). In this article, we address the performance challenges of LLMs for few-shot biomedical NER by investigating a dynamic prompting strategy involving retrieval-augmented generation (RAG). In our approach, the annotated in-context learning examples are selected based on their similarities with the input texts, and the prompt is dynamically updated for each instance during inference. We implemented and optimized static and dynamic prompt engineering techniques and evaluated them on five biomedical NER datasets. Static prompting with structured components increased average F1-scores by 12% for GPT-4, and 11% for GPT-3.5 and LLaMA 3-70B, relative to basic static prompting. Dynamic prompting further improved performance, with TF-IDF and SBERT retrieval methods yielding the best results, improving average F1-scores by 7.3% and 5.6% in 5-shot and 10-shot settings, respectively. These findings highlight the utility of contextually adaptive prompts via RAG for biomedical NER.

Related papers

• APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection [60.504476571531]
  Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.<n>Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data.<n>We introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimize the data usage alongside the prompt search.
  arXiv  Detail & Related papers  (2026-06-09T21:22:06Z)
• Generating Natural-Language Surgical Feedback: From Structured Representation to Domain-Grounded Evaluation [66.7752700084159]
  High-quality feedback from a surgical trainer is pivotal for improving trainee performance and long-term skill acquisition.<n>We present a structure-aware pipeline that learns a surgical action ontology from real trainer-to-trainee transcripts.
  arXiv  Detail & Related papers  (2025-11-19T06:19:34Z)
• Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning [49.559151128219725]
  Large language models (LLMs) have shown great promise in the medical domain, achieving strong performance on several benchmarks.<n>However, they continue to underperform in real-world medical scenarios, which often demand stronger context-awareness.<n>We propose Multifaceted Self-Refinement (MuSeR), a data-driven approach that enhances LLMs' context-awareness along three key facets.
  arXiv  Detail & Related papers  (2025-11-13T08:13:23Z)
• Enhancing Biomedical Named Entity Recognition using GLiNER-BioMed with Targeted Dictionary-Based Post-processing for BioASQ 2025 task 6 [0.0]
  This study evaluates the GLiNER-BioMed model on a BioASQ dataset.<n>We introduce a targeted dictionary-based post-processing strategy to address common misclassifications.<n>This work highlights the potential of dictionary-based refinement for pre-trained BioNER models but underscores the critical challenge of overfitting to development data.
  arXiv  Detail & Related papers  (2025-10-03T18:35:04Z)
• LLM-based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs [4.262074310505135]
  This paper explores prompt-based medical entity recognition using large language models (LLMs)<n>GPT-4o with prompt ensemble achieved the highest classification performance with an F1-score of 0.95 and recall of 0.98.<n>The ensemble method improved reliability by aggregating outputs through embedding-based similarity and majority voting.
  arXiv  Detail & Related papers  (2025-05-13T16:11:29Z)
• MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning [16.59236233944259]
  We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework.<n>MMRAG integrates four retrieval strategies: Random Mode, Top Mode, Diversity Mode, and Class Mode.<n>This study evaluates MMRAG on three core biomedical NLP tasks.
  arXiv  Detail & Related papers  (2025-02-21T21:36:48Z)
• Augmenting Biomedical Named Entity Recognition with General-domain Resources [47.24727904076347]
  Training a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations.<n>We propose GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training.<n>We systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances.
  arXiv  Detail & Related papers  (2024-06-15T15:28:02Z)
• BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine [19.861178160437827]
  Large Language Models (LLMs) have swiftly emerged as vital resources for different applications in the biomedical and healthcare domains. textscBiomedRAG attains superior performance across 5 biomedical NLP tasks. textscBiomedRAG outperforms other triple extraction systems with micro-F1 scores of 81.42 and 88.83 on GIT and ChemProt corpora, respectively.
  arXiv  Detail & Related papers  (2024-05-01T12:01:39Z)
• Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need [24.672621081551675]
  We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM. The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM.
  arXiv  Detail & Related papers  (2023-12-11T04:00:26Z)
• SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation [51.881877192924414]
  Sub-word segmentation is an essential pre-processing step for Neural Machine Translation (NMT) This paper introduces SelfSeg, a self-supervised neural sub-word segmentation method. SelfSeg is much faster to train/decode and requires only monolingual dictionaries instead of parallel corpora.
  arXiv  Detail & Related papers  (2023-07-31T04:38:47Z)
• BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks [68.39821375903591]
  Generalist AI holds the potential to address limitations due to its versatility in interpreting different data types. Here, we propose BiomedGPT, the first open-source and lightweight vision-language foundation model.
  arXiv  Detail & Related papers  (2023-05-26T17:14:43Z)
• An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
  The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians. Recent studies have achieved promising results in automatic impression generation using large-scale medical text data. These models often require substantial amounts of medical text data and have poor generalization performance.
  arXiv  Detail & Related papers  (2023-04-17T17:13:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.