Contextualized Medication Information Extraction Using Transformer-based Deep Learning Architectures

• URL: http://arxiv.org/abs/2303.08259v1
• Date: Tue, 14 Mar 2023 22:22:28 GMT
• Title: Contextualized Medication Information Extraction Using Transformer-based Deep Learning Architectures
• Authors: Aokun Chen, Zehao Yu, Xi Yang, Yi Guo, Jiang Bian, Yonghui Wu
• Abstract summary: We developed NLP systems for medication mention extraction, event classification (indicating medication changes discussed or not), and context classification. We explored 6 state-of-the-art pretrained transformer models for the three subtasks, including GatorTron, a large language model pretrained using >90 billion words of text. Our GatorTron models achieved the best F1-scores of 0.9828 for medication extraction (ranked 3rd), 0.9379 for event classification (ranked 2nd), and the best micro-average accuracy of 0.9126 for context classification.
• Score: 35.65283211002216
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Objective: To develop a natural language processing (NLP) system to extract medications and contextual information that help understand drug changes. This project is part of the 2022 n2c2 challenge. Materials and methods: We developed NLP systems for medication mention extraction, event classification (indicating medication changes discussed or not), and context classification to classify medication changes context into 5 orthogonal dimensions related to drug changes. We explored 6 state-of-the-art pretrained transformer models for the three subtasks, including GatorTron, a large language model pretrained using >90 billion words of text (including >80 billion words from >290 million clinical notes identified at the University of Florida Health). We evaluated our NLP systems using annotated data and evaluation scripts provided by the 2022 n2c2 organizers. Results:Our GatorTron models achieved the best F1-scores of 0.9828 for medication extraction (ranked 3rd), 0.9379 for event classification (ranked 2nd), and the best micro-average accuracy of 0.9126 for context classification. GatorTron outperformed existing transformer models pretrained using smaller general English text and clinical text corpora, indicating the advantage of large language models. Conclusion: This study demonstrated the advantage of using large transformer models for contextual medication information extraction from clinical narratives.

Related papers

• Customizing General-Purpose Foundation Models for Medical Report Generation [64.31265734687182]
  The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks. We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-06-09T03:02:36Z)
• Multimodal Model with Text and Drug Embeddings for Adverse Drug Reaction Classification [9.339007998235378]
  We introduce a multimodal model with two components. These components are state-of-the-art BERT-based models for language understanding and molecular property prediction. Experiments show that the molecular information obtained from neural networks is more beneficial for ADE classification than traditional molecular descriptors.
  arXiv  Detail & Related papers  (2022-10-21T11:41:45Z)
• Application of Deep Learning in Generating Structured Radiology Reports: A Transformer-Based Technique [0.4549831511476247]
  Natural language processing techniques can facilitate automatic information extraction and transformation of free-text formats to structured data. Deep learning (DL)-based models have been adapted for NLP experiments with promising results. In this study, we propose a transformer-based fine-grained named entity recognition architecture for clinical information extraction.
  arXiv  Detail & Related papers  (2022-09-25T08:03:15Z)
• Learning structures of the French clinical language:development and validation of word embedding models using 21 million clinical reports from electronic health records [2.5709272341038027]
  Methods based on transfer learning using pre-trained language models have achieved state-of-the-art results in most NLP applications. We aimed to evaluate the impact of adapting a language model to French clinical reports on downstream medical NLP tasks.
  arXiv  Detail & Related papers  (2022-07-26T14:46:34Z)
• Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [116.87918100031153]
  We propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG) CGT injects clinical relation triples into the visual features as prior knowledge to drive the decoding procedure. Experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods.
  arXiv  Detail & Related papers  (2022-06-04T13:16:30Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records [22.652798872046283]
  There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records ( EHRs) There are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters. It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs.
  arXiv  Detail & Related papers  (2022-02-02T14:28:51Z)
• Vision Transformers for femur fracture classification [59.99241204074268]
  The Vision Transformer (ViT) was able to correctly predict 83% of the test images. Good results were obtained in sub-fractures with the largest and richest dataset ever.
  arXiv  Detail & Related papers  (2021-08-07T10:12:42Z)
• Text Mining to Identify and Extract Novel Disease Treatments From Unstructured Datasets [56.38623317907416]
  We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the text. Our model successfully identified that Omeprazole can help treat heartburn.
  arXiv  Detail & Related papers  (2020-10-22T19:52:49Z)
• Med7: a transferable clinical natural language processing model for electronic health records [6.935142529928062]
  We introduce a named-entity recognition model for clinical natural language processing. The model is trained to recognise seven categories: drug names, route, frequency, dosage, strength, form, duration. We evaluate the transferability of the developed model using the data from the Intensive Care Unit in the US to secondary care mental health records (CRIS) in the UK.
  arXiv  Detail & Related papers  (2020-03-03T00:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.