Enhancing Vaccine Safety Surveillance: Extracting Vaccine Mentions from Emergency Department Triage Notes Using Fine-Tuned Large Language Models

• URL: http://arxiv.org/abs/2507.07599v1
• Date: Thu, 10 Jul 2025 09:57:08 GMT
• Title: Enhancing Vaccine Safety Surveillance: Extracting Vaccine Mentions from Emergency Department Triage Notes Using Fine-Tuned Large Language Models
• Authors: Sedigh Khademi, Jim Black, Christopher Palmer, Muhammad Javed, Hazel Clothier, Jim Buttery, Gerardo Luis Dimaguila,
• Abstract summary: The performance of prompt-engineered models, fine-tuned models, and a rule-based approach was compared.<n>The fine-tuned Llama 3 billion parameter model outperformed other models in its accuracy of extracting vaccine names.
• Score: 0.5025737475817937
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This study evaluates fine-tuned Llama 3.2 models for extracting vaccine-related information from emergency department triage notes to support near real-time vaccine safety surveillance. Prompt engineering was used to initially create a labeled dataset, which was then confirmed by human annotators. The performance of prompt-engineered models, fine-tuned models, and a rule-based approach was compared. The fine-tuned Llama 3 billion parameter model outperformed other models in its accuracy of extracting vaccine names. Model quantization enabled efficient deployment in resource-constrained environments. Findings demonstrate the potential of large language models in automating data extraction from emergency department notes, supporting efficient vaccine safety surveillance and early detection of emerging adverse events following immunization issues.

Related papers

• Actively evaluating and learning the distinctions that matter: Vaccine safety signal detection from emergency triage notes [0.5025737475817937]
  COVID-19 vaccines have showcased the global communitys ability to combat infectious diseases.<n>The need for post-licensure surveillance systems has grown due to the limited window for safety data collection in clinical trials and early widespread implementation.<n>This study aims to employ Natural Language Processing techniques and Active Learning to rapidly develop a classifier that detects potential vaccine safety issues from emergency department notes.
  arXiv  Detail & Related papers  (2025-07-24T06:18:34Z)
• Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat Falsehoods [4.6697477379475005]
  Generative AI models often learn and reproduce false information present in their training corpora.<n>This paper argues that AI models should be fine tuned on small, quarantined sets of explicitly labeled falsehoods as a "vaccine" against misinformation.
  arXiv  Detail & Related papers  (2025-05-23T13:20:23Z)
• Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models [3.582740629030056]
  An adjuvant is a chemical incorporated into vaccines that enhances their efficacy by improving the immune response.<n>This study explores the automated recognition of vaccine adjuvant names using Large Language Models (LLMs), specifically Generative Pretrained Transformers (GPT) and Large Language Model Meta AI (Llama)
  arXiv  Detail & Related papers  (2025-02-12T06:30:31Z)
• Immunogenicity Prediction with Dual Attention Enables Vaccine Target Selection [6.949493332885247]
  We introduce VenusVaccine, a novel deep learning solution for predicting immunogenicity in vaccines.<n>We also compile the most comprehensive immunogenicity dataset to date, encompassing over 7000 antigen sequences, structures, and immunogenicity labels from bacteria, virus, and tumor.<n>Our work provides an effective tool for vaccine design and sets valuable benchmarks for future research.
  arXiv  Detail & Related papers  (2024-10-03T16:33:35Z)
• Model X-ray:Detecting Backdoored Models via Decision Boundary [62.675297418960355]
  Backdoor attacks pose a significant security vulnerability for deep neural networks (DNNs) We propose Model X-ray, a novel backdoor detection approach based on the analysis of illustrated two-dimensional (2D) decision boundaries. Our approach includes two strategies focused on the decision areas dominated by clean samples and the concentration of label distribution.
  arXiv  Detail & Related papers  (2024-02-27T12:42:07Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• Safe AI for health and beyond -- Monitoring to transform a health service [51.8524501805308]
  We will assess the infrastructure required to monitor the outputs of a machine learning algorithm. We will present two scenarios with examples of monitoring and updates of models.
  arXiv  Detail & Related papers  (2023-03-02T17:27:45Z)
• Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification [60.49594822215981]
  This paper presents a classification model for detecting COVID-19 vaccination related search queries. We propose a novel approach of considering dense features as memory tokens that the model can attend to. We show that this new modeling approach enables a significant improvement to the Vaccine Search Insights (VSI) task.
  arXiv  Detail & Related papers  (2022-12-16T13:57:41Z)
• Improving the Adversarial Robustness of NLP Models by Information Bottleneck [112.44039792098579]
  Non-robust features can be easily manipulated by adversaries to fool NLP models. In this study, we explore the feasibility of capturing task-specific robust features, while eliminating the non-robust ones by using the information bottleneck theory. We show that the models trained with our information bottleneck-based method are able to achieve a significant improvement in robust accuracy.
  arXiv  Detail & Related papers  (2022-06-11T12:12:20Z)
• Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media [40.61499595293957]
  We propose a novel semi-supervised approach for vaccine attitude detection, called VADet. VADet is able to learn disentangled stance and aspect topics, and outperforms existing aspect-based sentiment analysis models on both stance detection and tweet clustering.
  arXiv  Detail & Related papers  (2022-05-06T15:24:33Z)
• An Optimal Control Approach to Learning in SIDARTHE Epidemic model [67.22168759751541]
  We propose a general approach for learning time-variant parameters of dynamic compartmental models from epidemic data. We forecast the epidemic evolution in Italy and France.
  arXiv  Detail & Related papers  (2020-10-28T10:58:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.