Meta Learning for Few-Shot Medical Text Classification
- URL: http://arxiv.org/abs/2212.01552v1
- Date: Sat, 3 Dec 2022 06:46:52 GMT
- Title: Meta Learning for Few-Shot Medical Text Classification
- Authors: Pankaj Sharma, Imran Qureshi, and Minh Tran
- Abstract summary: We investigate the use of meta-learning and robustness techniques on a broad corpus of benchmark text and medical data.
We find that meta-learning on text is a suitable framework for text-based data, providing better data efficiency and comparable performance to few-shot language models.
- Score: 1.2617078020344619
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Medical professionals frequently work in a data constrained setting to
provide insights across a unique demographic. A few medical observations, for
instance, informs the diagnosis and treatment of a patient. This suggests a
unique setting for meta-learning, a method to learn models quickly on new
tasks, to provide insights unattainable by other methods. We investigate the
use of meta-learning and robustness techniques on a broad corpus of benchmark
text and medical data. To do this, we developed new data pipelines, combined
language models with meta-learning approaches, and extended existing
meta-learning algorithms to minimize worst case loss. We find that
meta-learning on text is a suitable framework for text-based data, providing
better data efficiency and comparable performance to few-shot language models
and can be successfully applied to medical note data. Furthermore,
meta-learning models coupled with DRO can improve worst case loss across
disease codes.
Related papers
- LoGra-Med: Long Context Multi-Graph Alignment for Medical Vision-Language Model [55.80651780294357]
State-of-the-art medical multi-modal large language models (med-MLLM) leverage instruction-following data in pre-training.
LoGra-Med is a new multi-graph alignment algorithm that enforces triplet correlations across image modalities, conversation-based descriptions, and extended captions.
Our results show LoGra-Med matches LLAVA-Med performance on 600K image-text pairs for Medical VQA and significantly outperforms it when trained on 10% of the data.
arXiv Detail & Related papers (2024-10-03T15:52:03Z) - PneumoLLM: Harnessing the Power of Large Language Model for Pneumoconiosis Diagnosis [16.90918546157292]
Large language models (LLMs) have exhibits unprecedented ability when conducting multiple tasks in dialogue.
We propose a more effective method for harnessing LLMs in diagnosis with fewer learnable parameters.
We introduce the contextual multi-token engine and the information emitter module.
arXiv Detail & Related papers (2023-12-06T13:31:52Z) - Interpretable Medical Diagnostics with Structured Data Extraction by
Large Language Models [59.89454513692417]
Tabular data is often hidden in text, particularly in medical diagnostic reports.
We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM.
We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
arXiv Detail & Related papers (2023-06-08T09:12:28Z) - An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians.
Recent studies have achieved promising results in automatic impression generation using large-scale medical text data.
These models often require substantial amounts of medical text data and have poor generalization performance.
arXiv Detail & Related papers (2023-04-17T17:13:42Z) - Vision-Language Modelling For Radiological Imaging and Reports In The
Low Data Regime [70.04389979779195]
This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space.
We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains.
Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
arXiv Detail & Related papers (2023-03-30T18:20:00Z) - BERT WEAVER: Using WEight AVERaging to enable lifelong learning for
transformer-based models in biomedical semantic search engines [49.75878234192369]
We present WEAVER, a simple, yet efficient post-processing method that infuses old knowledge into the new model.
We show that applying WEAVER in a sequential manner results in similar word embedding distributions as doing a combined training on all data at once.
arXiv Detail & Related papers (2022-02-21T10:34:41Z) - MMLN: Leveraging Domain Knowledge for Multimodal Diagnosis [10.133715767542386]
We propose a knowledge-driven and data-driven framework for lung disease diagnosis.
We formulate diagnosis rules according to authoritative clinical medicine guidelines and learn the weights of rules from text data.
A multimodal fusion consisting of text and image data is designed to infer the marginal probability of lung disease.
arXiv Detail & Related papers (2022-02-09T04:12:30Z) - Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype
Prediction [55.94378672172967]
We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients.
We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks.
Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
arXiv Detail & Related papers (2020-09-02T02:50:30Z) - Predicting Scores of Medical Imaging Segmentation Methods with
Meta-Learning [0.30458514384586394]
We investigate meta-learning for segmentation across ten datasets of different organs and modalities.
We use support vector regression and deep neural networks to learn the relationship between the meta-features and prior model performance.
These results demonstrate the potential of meta-learning in medical imaging.
arXiv Detail & Related papers (2020-05-08T07:47:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.