Language Models are Few-shot Learners for Prognostic Prediction

• URL: http://arxiv.org/abs/2302.12692v4
• Date: Thu, 4 May 2023 18:04:49 GMT
• Title: Language Models are Few-shot Learners for Prognostic Prediction
• Authors: Zekai Chen, Mariann Micsinai Balan, Kevin Brown
• Abstract summary: We explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes.
• Score: 0.4254099382808599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clinical prediction is an essential task in the healthcare industry. However, the recent success of transformers, on which large language models are built, has not been extended to this domain. In this research, we explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. This paper investigates the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and addresses the challenge of few-shot learning in predicting rare disease areas. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes. The results demonstrate significant improvements in accuracy and highlight the potential of NLP in clinical research to improve early detection and intervention for different diseases.

Related papers

• Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction [10.403187385041702]
  We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue.
  arXiv  Detail & Related papers  (2025-01-28T22:38:45Z)
• Prediction of Lung Metastasis from Hepatocellular Carcinoma using the SEER Database [0.9055332067000195]
  Hepatocellular carcinoma (HCC) is a leading cause of cancer-related mortality. predictive models for lung metastasis inHCC remain limited in scope and clinical applicability. We develop and validate an end-to-end machine learning pipeline using data from the Surveillance, Epidemiology, and End Results (SEER) database.
  arXiv  Detail & Related papers  (2025-01-20T20:06:31Z)
• Almanac: Retrieval-Augmented Language Models for Clinical Medicine [1.5505279143287174]
  We develop Almanac, a large language model framework augmented with retrieval capabilities for medical guideline and treatment recommendations. Performance on a novel dataset of clinical scenarios evaluated by a panel of 5 board-certified and resident physicians demonstrates significant increases in factuality.
  arXiv  Detail & Related papers  (2023-03-01T02:30:11Z)
• This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Text [56.32427751440426]
  In clinical practice such models must not only be accurate, but provide doctors with interpretable and helpful results. We introduce ProtoPatient, a novel method based on prototypical networks and label-wise attention. We evaluate the model on two publicly available clinical datasets and show that it outperforms existing baselines.
  arXiv  Detail & Related papers  (2022-10-16T10:12:07Z)
• Cross-Lingual Knowledge Transfer for Clinical Phenotyping [55.92262310716537]
  We investigate cross-lingual knowledge transfer strategies to execute this task for clinics that do not use the English language. We evaluate these strategies for a Greek and a Spanish clinic leveraging clinical notes from different clinical domains. Our results show that using multilingual data overall improves clinical phenotyping models and can compensate for data sparseness.
  arXiv  Detail & Related papers  (2022-08-03T08:33:21Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• PathologyBERT -- Pre-trained Vs. A New Transformer Language Model for Pathology Domain [2.3628956573813498]
  Successful text mining of a large pathology database can play a critical role to advance 'big data' cancer research. No pathology-specific language space exist to support the rapid data-mining development in pathology space. PathologyBERT is a pre-trained masked language model which was trained on 347,173 histopathology specimen reports.
  arXiv  Detail & Related papers  (2022-05-13T20:42:07Z)
• Literature-Augmented Clinical Outcome Prediction [10.46990394710927]
  We introduce techniques to help bridge this gap between EBM and AI-based clinical models. We propose a novel system that automatically retrieves patient-specific literature based on intensive care (ICU) patient information. Our model is able to substantially boost predictive accuracy on three challenging tasks in comparison to strong recent baselines.
  arXiv  Detail & Related papers  (2021-11-16T11:19:02Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks [62.9447303059342]
  We show the importance of this problem in medical community. We present a modification of Bidirectional Representations from Transformers (BERT) model for classification sequence. We use a large-scale Russian EHR dataset consisting of about 4 million unique patient visits.
  arXiv  Detail & Related papers  (2020-07-15T09:22:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.