Use GPT-J Prompt Generation with RoBERTa for NER Models on Diagnosis Extraction of Periodontal Diagnosis from Electronic Dental Records

• URL: http://arxiv.org/abs/2311.10810v1
• Date: Fri, 17 Nov 2023 18:14:08 GMT
• Title: Use GPT-J Prompt Generation with RoBERTa for NER Models on Diagnosis Extraction of Periodontal Diagnosis from Electronic Dental Records
• Authors: Yao-Shun Chuang, Xiaoqian Jiang, Chun-Teh Lee, Ryan Brandon, Duong Tran, Oluwabunmi Tokede, Muhammad F. Walji
• Abstract summary: The prompt generation by GPT-J models was utilized to test the gold standard and to generate the seed. The performance revealed consistency, 0.92-0.97 in the F1 score, in all settings after training with the RoBERTa model.
• Score: 6.636721448099117
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study explored the usability of prompt generation on named entity recognition (NER) tasks and the performance in different settings of the prompt. The prompt generation by GPT-J models was utilized to directly test the gold standard as well as to generate the seed and further fed to the RoBERTa model with the spaCy package. In the direct test, a lower ratio of negative examples with higher numbers of examples in prompt achieved the best results with a F1 score of 0.72. The performance revealed consistency, 0.92-0.97 in the F1 score, in all settings after training with the RoBERTa model. The study highlighted the importance of seed quality rather than quantity in feeding NER models. This research reports on an efficient and accurate way to mine clinical notes for periodontal diagnoses, allowing researchers to easily and quickly build a NER model with the prompt generation approach.

Related papers

• Extracting periodontitis diagnosis in clinical notes with RoBERTa and regular expression [6.636721448099117]
  Two levels of complexity of regular expression (RE) methods were used to extract and generate the training data. The SpaCy package and RoBERTa transformer models were used to build the NER model and evaluate its performance with the manual-labeled gold standards. The NER models demonstrated excellent predictions, with the simple RE method showing 0.84-0.92 in the evaluation metrics, and the advanced and combined RE method demonstrating 0.95-0.99 in the evaluation.
  arXiv  Detail & Related papers  (2023-11-17T18:09:21Z)
• DiffNAS: Bootstrapping Diffusion Models by Prompting for Better Architectures [63.12993314908957]
  We propose a base model search approach, denoted "DiffNAS" We leverage GPT-4 as a supernet to expedite the search, supplemented with a search memory to enhance the results. Rigorous experimentation corroborates that our algorithm can augment the search efficiency by 2 times under GPT-based scenarios.
  arXiv  Detail & Related papers  (2023-10-07T09:10:28Z)
• Large Language Models to Identify Social Determinants of Health in Electronic Health Records [2.168737004368243]
  Social determinants of health (SDoH) have an important impact on patient outcomes but are incompletely collected from the electronic health records (EHRs) This study researched the ability of large language models to extract SDoH from free text in EHRs, where they are most commonly documented. 800 patient notes were annotated for SDoH categories, and several transformer-based models were evaluated.
  arXiv  Detail & Related papers  (2023-08-11T19:18:35Z)
• TREEMENT: Interpretable Patient-Trial Matching via Personalized Dynamic Tree-Based Memory Network [54.332862955411656]
  Clinical trials are critical for drug development but often suffer from expensive and inefficient patient recruitment. In recent years, machine learning models have been proposed for speeding up patient recruitment via automatically matching patients with clinical trials. We introduce a dynamic tree-based memory network model named TREEMENT to provide accurate and interpretable patient trial matching.
  arXiv  Detail & Related papers  (2023-07-19T12:35:09Z)
• Identifying and Extracting Rare Disease Phenotypes with Large Language Models [12.555067118549347]
  ChatGPT is a revolutionary large language model capable of following complex human prompts and generating high-quality responses. We compared its performance to the traditional fine-tuning approach and conducted an in-depth error analysis. ChatGPT achieved similar or higher accuracy for certain entities (i.e., rare diseases and signs) in the one-shot setting.
  arXiv  Detail & Related papers  (2023-06-22T03:52:12Z)
• Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering [20.534197056683695]
  This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks. We developed a task-specific prompt framework that includes baseline prompts, annotation guideline-based prompts, error analysis-based instructions, and annotated samples. We assessed each prompt's effectiveness and compared the models to BioClinicalBERT.
  arXiv  Detail & Related papers  (2023-03-29T02:46:18Z)
• Clinical Deterioration Prediction in Brazilian Hospitals Based on Artificial Neural Networks and Tree Decision Models [56.93322937189087]
  An extremely boosted neural network (XBNet) is used to predict clinical deterioration (CD) The XGBoost model obtained the best results in predicting CD among Brazilian hospitals' data.
  arXiv  Detail & Related papers  (2022-12-17T23:29:14Z)
• Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition [8.344476599818826]
  We introduce NRB, a new testbed designed to diagnose Name Regularity Bias of NER models. Our results indicate that all state-of-the-art models we tested show such a bias. We propose a novel model-agnostic training method that adds learnable adversarial noise to some entity mentions.
  arXiv  Detail & Related papers  (2021-07-24T13:55:35Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.