DictLLM: Harnessing Key-Value Data Structures with Large Language Models
for Enhanced Medical Diagnostics
- URL: http://arxiv.org/abs/2402.11481v1
- Date: Sun, 18 Feb 2024 07:10:02 GMT
- Title: DictLLM: Harnessing Key-Value Data Structures with Large Language Models
for Enhanced Medical Diagnostics
- Authors: YiQiu Guo, Yuchen Yang, Ya Zhang, Yu Wang, Yanfeng Wang
- Abstract summary: DictLLM is an innovative framework designed to improve the modeling of key-value structured data, like medical laboratory reports, for generating medical diagnoses.
We carry out experiments using various LLM models on a comprehensive real-world medical laboratory report dataset for automatic diagnosis generation.
- Score: 36.057925881268226
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Structured data offers a sophisticated mechanism for the organization of
information. Existing methodologies for the text-serialization of structured
data in the context of large language models fail to adequately address the
heterogeneity inherent in key-value structured data. These methods are not
ideal and frequently result in larger input sizes and poor adaptability to
input changes. In this paper, we introduce DictLLM, an innovative framework
designed to improve the modeling of key-value structured data, like medical
laboratory reports, for generating medical diagnoses. DictLLM integrates three
key components: (1) group positional encoding to maintain permutation
invariance, (2) hierarchical attention bias to capture the inherent bias in
structured data, and (3) an optimal transport alignment layer that aligns the
embedding generated by the dictionary encoder with the LLM, thereby producing a
sequence of fixed-length virtual tokens. We carry out experiments using various
LLM models on a comprehensive real-world medical laboratory report dataset for
automatic diagnosis generation, our findings illustrate that DictLLM
significantly outperforms established baseline methods and few-shot GPT-4
implementations in terms of both Rouge-L and Knowledge F1 scores. Furthermore,
our evaluation of the framework's scalability and robustness, through a series
of experiments, underscores its exceptional capability in accurately modeling
the complex key-value data structure of medical dictionary data.
Related papers
- MEDFuse: Multimodal EHR Data Fusion with Masked Lab-Test Modeling and Large Language Models [11.798375238713488]
MEDFuse is a framework that integrates structured and unstructured medical data.
It achieves over 90% F1 score in the 10-disease multi-label classification task.
arXiv Detail & Related papers (2024-07-17T04:17:09Z) - Local Large Language Models for Complex Structured Medical Tasks [0.0]
This paper introduces an approach that combines the language reasoning capabilities of large language models with the benefits of local training to tackle complex, domain-specific tasks.
Specifically, the authors demonstrate their approach by extracting structured condition codes from pathology reports.
arXiv Detail & Related papers (2023-08-03T12:36:13Z) - Interpretable Medical Diagnostics with Structured Data Extraction by
Large Language Models [59.89454513692417]
Tabular data is often hidden in text, particularly in medical diagnostic reports.
We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM.
We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
arXiv Detail & Related papers (2023-06-08T09:12:28Z) - An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians.
Recent studies have achieved promising results in automatic impression generation using large-scale medical text data.
These models often require substantial amounts of medical text data and have poor generalization performance.
arXiv Detail & Related papers (2023-04-17T17:13:42Z) - Two heads are better than one: Enhancing medical representations by
pre-training over structured and unstructured electronic health records [23.379185792773875]
We propose a unified deep learning-based medical pre-trained language model, named UMM-PLM, to automatically learn representative features from multimodal EHRs.
We first developed parallel unimodal information representation modules to capture the unimodal-specific characteristic, where unimodal representations were learned from each data source separately.
A cross-modal module was further introduced to model the interactions between different modalities.
arXiv Detail & Related papers (2022-01-25T06:14:49Z) - A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding.
These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information.
Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
arXiv Detail & Related papers (2021-02-26T17:49:58Z) - SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation.
Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
arXiv Detail & Related papers (2021-01-02T01:15:57Z) - Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
We investigate three schemes to improve the model generalization ability for few-shot settings.
We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data.
We create new state-of-the-art results on both few-shot and training-free settings.
arXiv Detail & Related papers (2020-12-29T23:43:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.