A Multi-View Joint Learning Framework for Embedding Clinical Codes and Text Using Graph Neural Networks

• URL: http://arxiv.org/abs/2301.11608v1
• Date: Fri, 27 Jan 2023 09:19:03 GMT
• Title: A Multi-View Joint Learning Framework for Embedding Clinical Codes and Text Using Graph Neural Networks
• Authors: Lecheng Kong, Christopher King, Bradley Fritz, Yixin Chen
• Abstract summary: We propose a framework that learns from codes and text to combine the availability and forward-looking nature of text and better performance of ICD codes. Our approach uses a Graph Neural Network (GNN) to process ICD codes, and Bi-LSTM to process text. In experiments using planned surgical procedure text, our model outperforms BERT models fine-tuned to clinical data.
• Score: 23.06795121693656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning to represent free text is a core task in many clinical machine learning (ML) applications, as clinical text contains observations and plans not otherwise available for inference. State-of-the-art methods use large language models developed with immense computational resources and training data; however, applying these models is challenging because of the highly varying syntax and vocabulary in clinical free text. Structured information such as International Classification of Disease (ICD) codes often succinctly abstracts the most important facts of a clinical encounter and yields good performance, but is often not as available as clinical text in real-world scenarios. We propose a \textbf{multi-view learning framework} that jointly learns from codes and text to combine the availability and forward-looking nature of text and better performance of ICD codes. The learned text embeddings can be used as inputs to predictive algorithms independent of the ICD codes during inference. Our approach uses a Graph Neural Network (GNN) to process ICD codes, and Bi-LSTM to process text. We apply Deep Canonical Correlation Analysis (DCCA) to enforce the two views to learn a similar representation of each patient. In experiments using planned surgical procedure text, our model outperforms BERT models fine-tuned to clinical data, and in experiments using diverse text in MIMIC-III, our model is competitive to a fine-tuned BERT at a tiny fraction of its computational effort.

Related papers

• SNOBERT: A Benchmark for clinical notes entity linking in the SNOMED CT clinical terminology [43.89160296332471]
  We propose a method for linking text spans in clinical notes to specific concepts in the SNOMED CT using BERT-based models. The method consists of two stages: candidate selection and candidate matching. The models were trained on one of the largest publicly available dataset of labeled clinical notes.
  arXiv  Detail & Related papers  (2024-05-25T08:00:44Z)
• CoRelation: Boosting Automatic ICD Coding Through Contextualized Code Relation Learning [56.782963838838036]
  We propose a novel approach, a contextualized and flexible framework, to enhance the learning of ICD code representations. Our approach employs a dependent learning paradigm that considers the context of clinical notes in modeling all possible code relations.
  arXiv  Detail & Related papers  (2024-02-24T03:25:28Z)
• Preserving the knowledge of long clinical texts using aggregated ensembles of large language models [0.0]
  Clinical texts contain rich and valuable information that can be used for various clinical outcome prediction tasks. Applying large language models, such as BERT-based models, to clinical texts poses two major challenges. This paper proposes a novel method to preserve the knowledge of long clinical texts using aggregated ensembles of large language models.
  arXiv  Detail & Related papers  (2023-11-02T19:50:02Z)
• Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models [48.07083163501746]
  Clinical natural language processing requires methods that can address domain-specific challenges. We propose an innovative, resource-efficient approach, ClinGen, which infuses knowledge into the process. Our empirical study across 7 clinical NLP tasks and 16 datasets reveals that ClinGen consistently enhances performance across various tasks.
  arXiv  Detail & Related papers  (2023-11-01T04:37:28Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• Development and validation of a natural language processing algorithm to pseudonymize documents in the context of a clinical data warehouse [53.797797404164946]
  The study highlights the difficulties faced in sharing tools and resources in this domain. We annotated a corpus of clinical documents according to 12 types of identifying entities. We build a hybrid system, merging the results of a deep learning model as well as manual rules.
  arXiv  Detail & Related papers  (2023-03-23T17:17:46Z)
• ICDBigBird: A Contextual Embedding Model for ICD Code Classification [71.58299917476195]
  Contextual word embedding models have achieved state-of-the-art results in multiple NLP tasks. ICDBigBird is a BigBird-based model which can integrate a Graph Convolutional Network (GCN) Our experiments on a real-world clinical dataset demonstrate the effectiveness of our BigBird-based model on the ICD classification task.
  arXiv  Detail & Related papers  (2022-04-21T20:59:56Z)
• HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing [3.762895631262445]
  We developed a novel prompt-based clinical NLP framework called HealthPrompt. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
  arXiv  Detail & Related papers  (2022-03-09T21:44:28Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text [19.701824507057623]
  This paper proposes a Dilated Convolutional Attention Network (DCAN), integrating dilated convolutions, residual connections, and label attention, for medical code assignment. It adopts dilated convolutions to capture complex medical patterns with a receptive field which increases exponentially with dilation size.
  arXiv  Detail & Related papers  (2020-09-30T11:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.