Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks

• URL: http://arxiv.org/abs/2208.14923v1
• Date: Wed, 31 Aug 2022 15:36:27 GMT
• Title: Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks
• Authors: David Oniani, Sonish Sivarajkumar, Yanshan Wang
• Abstract summary: Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare. Deep learning has achieved state-of-the-art performance in many clinical NLP tasks. Training deep learning models usually require large annotated datasets, which are normally not publicly available.
• Score: 3.9586758145580014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that leverages a large amount of free-text data in electronic health records (EHRs) to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models usually require large annotated datasets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated datasets is typical in clinical NLP and therefore, ensuring that deep learning models perform well is crucial for the models to be used in real-world applications. A widely adopted approach is fine-tuning existing Pre-trained Language Models (PLMs), but these attempts fall short when the training dataset contains only a few annotated samples. Few-Shot Learning (FSL) has recently been investigated to tackle this problem. Siamese Neural Network (SNN) has been widely utilized as an FSL approach in computer vision, but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. In this paper, we propose two SNN-based FSL approaches for clinical NLP, including pre-trained SNN (PT-SNN) and SNN with second-order embeddings (SOE-SNN). We evaluated the proposed approaches on two clinical tasks, namely clinical text classification and clinical named entity recognition. We tested three few-shot settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP tasks were benchmarked using three PLMs, including BERT, BioBERT, and BioClinicalBERT. The experimental results verified the effectiveness of the proposed SNN-based FSL approaches in both clinical NLP tasks.

Related papers

• An Introduction to Natural Language Processing Techniques and Framework for Clinical Implementation in Radiation Oncology [1.2714439146420664]
  We present state-of-the-art NLP applications that employ large language models (LLMs) in radiation oncology research. LLMs are prone to many errors such as hallucinations, biases, and ethical violations, which necessitate rigorous evaluation and validation. Our article aims to provide guidance and insights for researchers and clinicians who are interested in developing and using NLP models in clinical radiation oncology.
  arXiv  Detail & Related papers  (2023-11-03T19:32:35Z)
• Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models [48.07083163501746]
  Clinical natural language processing requires methods that can address domain-specific challenges. We propose an innovative, resource-efficient approach, ClinGen, which infuses knowledge into the process. Our empirical study across 7 clinical NLP tasks and 16 datasets reveals that ClinGen consistently enhances performance across various tasks.
  arXiv  Detail & Related papers  (2023-11-01T04:37:28Z)
• An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing [4.758617742396169]
  We present a comprehensive and systematic experimental study on prompt engineering for five clinical NLP tasks. We assessed the prompts proposed in recent literature, including simple prefix, simple cloze, chain of thought, and anticipatory prompts. We provide novel insights and guidelines for prompt engineering for LLMs in clinical NLP.
  arXiv  Detail & Related papers  (2023-09-14T19:35:00Z)
• Multi-Site Clinical Federated Learning using Recursive and Attentive Models and NVFlare [13.176351544342735]
  This paper develops an integrated framework that addresses data privacy and regulatory compliance challenges. It includes the development of an integrated framework that addresses data privacy and regulatory compliance challenges while maintaining elevated accuracy and substantiating the efficacy of the proposed approach.
  arXiv  Detail & Related papers  (2023-06-28T17:00:32Z)
• Do We Still Need Clinical Language Models? [15.023633270864675]
  We show that relatively small specialized clinical models substantially outperform all in-context learning approaches. We release the code and the models used under the PhysioNet Credentialed Health Data license and data use agreement.
  arXiv  Detail & Related papers  (2023-02-16T05:08:34Z)
• Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
  Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community. The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
  arXiv  Detail & Related papers  (2022-07-01T14:17:11Z)
• HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing [3.762895631262445]
  We developed a novel prompt-based clinical NLP framework called HealthPrompt. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
  arXiv  Detail & Related papers  (2022-03-09T21:44:28Z)
• Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing [55.52858954615655]
  We conduct a systematic study on fine-tuning stability in biomedical NLP. We show that finetuning performance may be sensitive to pretraining settings, especially in low-resource domains. We show that these techniques can substantially improve fine-tuning performance for lowresource biomedical NLP applications.
  arXiv  Detail & Related papers  (2021-12-15T04:20:35Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Uncovering the structure of clinical EEG signals with self-supervised learning [64.4754948595556]
  Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG) By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
  arXiv  Detail & Related papers  (2020-07-31T14:34:47Z)
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing [73.37262264915739]
  We show that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains. Our experiments show that domain-specific pretraining serves as a solid foundation for a wide range of biomedical NLP tasks.
  arXiv  Detail & Related papers  (2020-07-31T00:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.