AE-GPT: Using Large Language Models to Extract Adverse Events from Surveillance Reports-A Use Case with Influenza Vaccine Adverse Events

• URL: http://arxiv.org/abs/2309.16150v1
• Date: Thu, 28 Sep 2023 03:53:21 GMT
• Title: AE-GPT: Using Large Language Models to Extract Adverse Events from Surveillance Reports-A Use Case with Influenza Vaccine Adverse Events
• Authors: Yiming Li, Jianfu Li, Jianping He, Cui Tao
• Abstract summary: Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. This study particularly focuses on AEs to evaluate LLMs' capability for AE extraction. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match.
• Score: 13.221548807536067
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Though Vaccines are instrumental in global health, mitigating infectious diseases and pandemic outbreaks, they can occasionally lead to adverse events (AEs). Recently, Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. Utilizing data from the Vaccine Adverse Event Reporting System (VAERS) from 1990 to 2016, this study particularly focuses on AEs to evaluate LLMs' capability for AE extraction. A variety of prevalent LLMs, including GPT-2, GPT-3 variants, GPT-4, and Llama 2, were evaluated using Influenza vaccine as a use case. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match. The encouraging performance of the AE-GPT underscores LLMs' potential in processing medical data, indicating a significant stride towards advanced AE detection, thus presumably generalizable to other AE extraction tasks.

Related papers

• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references. We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey. Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)
• Cancer Vaccine Adjuvant Name Recognition from Biomedical Literature using Large Language Models [3.582740629030056]
  An adjuvant is a chemical incorporated into vaccines that enhances their efficacy by improving the immune response. This study explores the automated recognition of vaccine adjuvant names using Large Language Models (LLMs), specifically Generative Pretrained Transformers (GPT) and Large Language Model Meta AI (Llama)
  arXiv  Detail & Related papers  (2025-02-12T06:30:31Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• Improving Entity Recognition Using Ensembles of Deep Learning and Fine-tuned Large Language Models: A Case Study on Adverse Event Extraction from Multiple Sources [13.750202656564907]
  Adverse event (AE) extraction is crucial for monitoring and analyzing the safety profiles of immunizations. This study aims to evaluate the effectiveness of large language models (LLMs) and traditional deep learning models in AE extraction.
  arXiv  Detail & Related papers  (2024-06-26T03:56:21Z)
• Event Detection from Social Media for Epidemic Prediction [76.90779562626541]
  We develop a framework to extract and analyze epidemic-related events from social media posts. Experimentation reveals how ED models trained on COVID-based SPEED can effectively detect epidemic events for three unseen epidemics. We show that reporting sharp increases in the extracted events by our framework can provide warnings 4-9 weeks earlier than the WHO epidemic declaration for Monkeypox.
  arXiv  Detail & Related papers  (2024-04-02T06:31:17Z)
• Unsupervised Anomaly Detection using Aggregated Normative Diffusion [46.24703738821696]
  Unsupervised anomaly detection has the potential to identify a broader spectrum of anomalies. Existing state-of-the-art UAD approaches do not generalise well to diverse types of anomalies. We introduce a new UAD method named Aggregated Normative Diffusion (ANDi)
  arXiv  Detail & Related papers  (2023-12-04T14:02:56Z)
• COVID-19 Detection from Exhaled Breath [0.4321423008988813]
  SARS-CoV-2 coronavirus emerged in 2019, causing a COVID-19 pandemic. In this paper, we introduce a cheap, fast, and non-invasive detection system, which exploits only the exhaled breath. Despite the simplicity of use, our system showed a performance comparable to the traditional polymerase-chain-reaction and antigen testing.
  arXiv  Detail & Related papers  (2023-05-30T17:01:53Z)
• Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification [60.49594822215981]
  This paper presents a classification model for detecting COVID-19 vaccination related search queries. We propose a novel approach of considering dense features as memory tokens that the model can attend to. We show that this new modeling approach enables a significant improvement to the Vaccine Search Insights (VSI) task.
  arXiv  Detail & Related papers  (2022-12-16T13:57:41Z)
• ANet: Autoencoder-Based Local Field Potential Feature Extractor for Evaluating An Antidepressant Effect in Mice after Administering Kratom Leaf Extracts [0.44325173792230727]
  We used an autoencoder (AE)-based anomaly detector called ANet to measure the similarity of mice's local field potential (LFP) features that responded to KT leave extracts and AD flu. The features that responded to KT syrup had the highest similarity to those that responded to the AD flu at 85.62 $pm$ 0.29%.
  arXiv  Detail & Related papers  (2022-09-17T01:14:26Z)
• Efficient Novelty Detection Methods for Early Warning of Potential Fatal Diseases [0.0]
  Fatal diseases, as Critical Health Episodes (CHEs), represent real dangers for patients hospitalized in Intensive Care Units. This study focused on building a highly effective early warning system for CHEs such as Acute Hypotensive Episodes and Tachycardia Episodes.
  arXiv  Detail & Related papers  (2022-08-06T19:04:51Z)
• Federated Learning Enables Big Data for Rare Cancer Boundary Detection [98.5549882883963]
  We present findings from the largest Federated ML study to-date, involving data from 71 healthcare institutions across 6 continents. We generate an automatic tumor boundary detector for the rare disease of glioblastoma. We demonstrate a 33% improvement over a publicly trained model to delineate the surgically targetable tumor, and 23% improvement over the tumor's entire extent.
  arXiv  Detail & Related papers  (2022-04-22T17:27:00Z)
• Preliminary Results from a Peer-Led, Social Network Intervention, Augmented by Artificial Intelligence to Prevent HIV among Youth Experiencing Homelessness [47.21347530335741]
  Each year, there are nearly 4 million youth experiencing homelessness in the United States with HIV prevalence ranging from 3 to 11.5%. PCA models for HIV prevention have been used successfully in many populations, but there have been notable failures. We tested a new PCA intervention for YEH, with three arms: (1) an arm using an artificial intelligence (AI) planning algorithm to select PCA, (2) a popularity arm--operationalized as highest degree centrality (DC), and (3) an observation only comparison group (OBS) Both the AI and DC arms showed improvements over time. AI-based PCA selection led to better
  arXiv  Detail & Related papers  (2020-07-11T02:17:53Z)
• Natural Language Processing with Deep Learning for Medical Adverse Event Detection from Free-Text Medical Narratives: A Case Study of Detecting Total Hip Replacement Dislocation [0.0]
  We propose deep learning based NLP (DL-NLP) models for efficient and accurate hip dislocation AE detection following total hip replacement. We benchmarked these proposed models with a wide variety of traditional machine learning based NLP (ML-NLP) models. All DL-NLP models out-performed all of the ML-NLP models, with a convolutional neural network (CNN) model achieving the best overall performance.
  arXiv  Detail & Related papers  (2020-04-17T16:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.