Language Models are Few-shot Learners for Prognostic Prediction

• URL: http://arxiv.org/abs/2302.12692v4
• Date: Thu, 4 May 2023 18:04:49 GMT
• Title: Language Models are Few-shot Learners for Prognostic Prediction
• Authors: Zekai Chen, Mariann Micsinai Balan, Kevin Brown
• Abstract summary: We explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes.
• Score: 0.4254099382808599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clinical prediction is an essential task in the healthcare industry. However, the recent success of transformers, on which large language models are built, has not been extended to this domain. In this research, we explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. This paper investigates the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and addresses the challenge of few-shot learning in predicting rare disease areas. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes. The results demonstrate significant improvements in accuracy and highlight the potential of NLP in clinical research to improve early detection and intervention for different diseases.

Related papers

• EEG-Language Modeling for Pathology Detection [0.0]
  This study pioneers EEG-language models trained on clinical reports and 15000 EEGs. Our results indicate that models learn richer representations from being exposed to a variety of report segments. representations of EEG-language models can significantly improve pathology detection compared to those of EEG-only models.
  arXiv  Detail & Related papers  (2024-09-02T10:03:03Z)
• CPLLM: Clinical Prediction with Large Language Models [0.07083082555458872]
  We present a method that involves fine-tuning a pre-trained Large Language Model (LLM) for clinical disease and readmission prediction. For diagnosis prediction, we predict whether patients will be diagnosed with a target disease during their next visit or in the subsequent diagnosis, leveraging their historical diagnosis records. Our experiments have shown that our proposed method, CPLLM, surpasses all the tested models in terms of PR-AUC and ROC-AUC metrics.
  arXiv  Detail & Related papers  (2023-09-20T13:24:12Z)
• Almanac: Retrieval-Augmented Language Models for Clinical Medicine [1.5505279143287174]
  We develop Almanac, a large language model framework augmented with retrieval capabilities for medical guideline and treatment recommendations. Performance on a novel dataset of clinical scenarios evaluated by a panel of 5 board-certified and resident physicians demonstrates significant increases in factuality.
  arXiv  Detail & Related papers  (2023-03-01T02:30:11Z)
• This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Text [56.32427751440426]
  In clinical practice such models must not only be accurate, but provide doctors with interpretable and helpful results. We introduce ProtoPatient, a novel method based on prototypical networks and label-wise attention. We evaluate the model on two publicly available clinical datasets and show that it outperforms existing baselines.
  arXiv  Detail & Related papers  (2022-10-16T10:12:07Z)
• Cross-Lingual Knowledge Transfer for Clinical Phenotyping [55.92262310716537]
  We investigate cross-lingual knowledge transfer strategies to execute this task for clinics that do not use the English language. We evaluate these strategies for a Greek and a Spanish clinic leveraging clinical notes from different clinical domains. Our results show that using multilingual data overall improves clinical phenotyping models and can compensate for data sparseness.
  arXiv  Detail & Related papers  (2022-08-03T08:33:21Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• PathologyBERT -- Pre-trained Vs. A New Transformer Language Model for Pathology Domain [2.3628956573813498]
  Successful text mining of a large pathology database can play a critical role to advance 'big data' cancer research. No pathology-specific language space exist to support the rapid data-mining development in pathology space. PathologyBERT is a pre-trained masked language model which was trained on 347,173 histopathology specimen reports.
  arXiv  Detail & Related papers  (2022-05-13T20:42:07Z)
• Literature-Augmented Clinical Outcome Prediction [10.46990394710927]
  We introduce techniques to help bridge this gap between EBM and AI-based clinical models. We propose a novel system that automatically retrieves patient-specific literature based on intensive care (ICU) patient information. Our model is able to substantially boost predictive accuracy on three challenging tasks in comparison to strong recent baselines.
  arXiv  Detail & Related papers  (2021-11-16T11:19:02Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.