Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT

• URL: http://arxiv.org/abs/2308.06294v2
• Date: Thu, 9 Nov 2023 15:18:38 GMT
• Title: Enhancing Phenotype Recognition in Clinical Notes Using Large Language Models: PhenoBCBERT and PhenoGPT
• Authors: Jingye Yang, Cong Liu, Wendy Deng, Da Wu, Chunhua Weng, Yunyun Zhou, Kai Wang
• Abstract summary: We developed two types of models: PhenoBCBERT, a BERT-based model, and PhenoGPT, a GPT-based model. We found that our methods can extract more phenotype concepts, including novel ones not characterized by HPO.
• Score: 11.20254354103518
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We hypothesize that large language models (LLMs) based on the transformer architecture can enable automated detection of clinical phenotype terms, including terms not documented in the HPO. In this study, we developed two types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT as its pre-trained model, and PhenoGPT, a GPT-based model that can be initialized from diverse GPT models, including open-source versions such as GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO recognition tool that combines rule-based and deep learning methods. We found that our methods can extract more phenotype concepts, including novel ones not characterized by HPO. We also performed case studies on biomedical literature to illustrate how new phenotype information can be recognized and extracted. We compared current BERT-based versus GPT-based models for phenotype tagging, in multiple aspects including model architecture, memory usage, speed, accuracy, and privacy protection. We also discussed the addition of a negation step and an HPO normalization layer to the transformer models for improved HPO term tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery of phenotype terms from clinical notes and biomedical literature, facilitating automated downstream tasks to derive new biological insights on human diseases.

Related papers

• High-Throughput Phenotyping of Clinical Text Using Large Language Models [0.0]
  GPT-4 surpasses GPT-3.5-Turbo in identifying, categorizing, and normalizing signs. GPT-4 results in high performance and generalizability across several phenotyping tasks.
  arXiv  Detail & Related papers  (2024-08-02T12:00:00Z)
• A Large Language Model Outperforms Other Computational Approaches to the High-Throughput Phenotyping of Physician Notes [0.0]
  This study compares three computational approaches to high- throughput phenotyping. A Large Language Model (LLM) incorporating generative AI, a Natural Language Processing (NLP) approach utilizing deep learning for span categorization, and a hybrid approach combining word vectors with machine learning. The approach that implemented GPT-4 (a Large Language Model) demonstrated superior performance.
  arXiv  Detail & Related papers  (2024-06-20T22:05:34Z)
• Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs [54.223394825528665]
  We develop an approach that uses lightweight adapter modules to inject structured biomedical knowledge into pre-trained language models. We use two large KGs, the biomedical knowledge system UMLS and the novel biochemical OntoChem, with two prominent biomedical PLMs, PubMedBERT and BioLinkBERT. We show that our methodology leads to performance improvements in several instances while keeping requirements in computing power low.
  arXiv  Detail & Related papers  (2023-12-21T14:26:57Z)
• An evaluation of GPT models for phenotype concept recognition [0.4715973318447338]
  We examine the performance of the latest Generative Pre-trained Transformer (GPT) models for clinical phenotyping and phenotype annotation. Our results show that, with an appropriate setup, these models can achieve state of the art performance.
  arXiv  Detail & Related papers  (2023-09-29T12:06:55Z)
• A statistical framework for GWAS of high dimensional phenotypes using summary statistics, with application to metabolite GWAS [1.6058099298620425]
  We develop a novel model, theoretical framework, and set of methods to perform Bayesian inference in GWAS of high dimensional phenotypes. We demonstrate the utility of our procedure by applying it to metabolite GWAS.
  arXiv  Detail & Related papers  (2023-03-17T19:33:25Z)
• Unsupervised ensemble-based phenotyping helps enhance the discoverability of genes related to heart morphology [57.25098075813054]
  We propose a new framework for gene discovery entitled Un Phenotype Ensembles. It builds a redundant yet highly expressive representation by pooling a set of phenotypes learned in an unsupervised manner. These phenotypes are then analyzed via (GWAS), retaining only highly confident and stable associations.
  arXiv  Detail & Related papers  (2023-01-07T18:36:44Z)
• Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing [55.52858954615655]
  We conduct a systematic study on fine-tuning stability in biomedical NLP. We show that finetuning performance may be sensitive to pretraining settings, especially in low-resource domains. We show that these techniques can substantially improve fine-tuning performance for lowresource biomedical NLP applications.
  arXiv  Detail & Related papers  (2021-12-15T04:20:35Z)
• Hybrid deep learning methods for phenotype prediction from clinical notes [4.866431869728018]
  This paper proposes a novel hybrid model for automatically extracting patient phenotypes using natural language processing and deep learning models. The proposed hybrid model is based on a neural bidirectional sequence model (BiLSTM or BiGRU) and a Convolutional Neural Network (CNN) for identifying patient's phenotypes in discharge reports.
  arXiv  Detail & Related papers  (2021-08-16T05:57:28Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study [62.376800537374024]
  We study scientific LMs for KG completion, exploring whether we can tap into their latent knowledge to enhance biomedical link prediction. We integrate the LM-based models with KG embedding models, using a router method that learns to assign each input example to either type of model and provides a substantial boost in performance.
  arXiv  Detail & Related papers  (2021-06-17T17:55:33Z)
• Energy-based View of Retrosynthesis [70.66156081030766]
  We propose a framework that unifies sequence- and graph-based methods as energy-based models. We present a novel dual variant within the framework that performs consistent training over Bayesian forward- and backward-prediction. This model improves state-of-the-art performance by 9.6% for template-free approaches where the reaction type is unknown.
  arXiv  Detail & Related papers  (2020-07-14T18:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.