DictLLM: Harnessing Key-Value Data Structures with Large Language Models for Enhanced Medical Diagnostics

• URL: http://arxiv.org/abs/2402.11481v1
• Date: Sun, 18 Feb 2024 07:10:02 GMT
• Title: DictLLM: Harnessing Key-Value Data Structures with Large Language Models for Enhanced Medical Diagnostics
• Authors: YiQiu Guo, Yuchen Yang, Ya Zhang, Yu Wang, Yanfeng Wang
• Abstract summary: DictLLM is an innovative framework designed to improve the modeling of key-value structured data, like medical laboratory reports, for generating medical diagnoses. We carry out experiments using various LLM models on a comprehensive real-world medical laboratory report dataset for automatic diagnosis generation.
• Score: 36.057925881268226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured data offers a sophisticated mechanism for the organization of information. Existing methodologies for the text-serialization of structured data in the context of large language models fail to adequately address the heterogeneity inherent in key-value structured data. These methods are not ideal and frequently result in larger input sizes and poor adaptability to input changes. In this paper, we introduce DictLLM, an innovative framework designed to improve the modeling of key-value structured data, like medical laboratory reports, for generating medical diagnoses. DictLLM integrates three key components: (1) group positional encoding to maintain permutation invariance, (2) hierarchical attention bias to capture the inherent bias in structured data, and (3) an optimal transport alignment layer that aligns the embedding generated by the dictionary encoder with the LLM, thereby producing a sequence of fixed-length virtual tokens. We carry out experiments using various LLM models on a comprehensive real-world medical laboratory report dataset for automatic diagnosis generation, our findings illustrate that DictLLM significantly outperforms established baseline methods and few-shot GPT-4 implementations in terms of both Rouge-L and Knowledge F1 scores. Furthermore, our evaluation of the framework's scalability and robustness, through a series of experiments, underscores its exceptional capability in accurately modeling the complex key-value data structure of medical dictionary data.

Related papers

• GENIE: Generative Note Information Extraction model for structuring EHR data [14.057531175321113]
  We introduce GENIE, a Generative Note Information Extraction system. GENIE processes entire paragraphs in a single pass, extracting entities, assertion statuses, locations, modifier, values, and purposes with high accuracy. Using a robust data preparation pipeline and fine-tuned small scale LLMs, GENIE achieves competitive performance across multiple information extraction tasks.
  arXiv  Detail & Related papers  (2025-01-30T15:42:24Z)
• Representation Learning of Structured Data for Medical Foundation Models [29.10129199884847]
  We introduce the UniStruct architecture to design a multimodal medical foundation model of unstructured text and structured data. Our approach is validated through model pre-training on both an extensive internal medical database and a public repository of structured medical records.
  arXiv  Detail & Related papers  (2024-10-17T09:02:28Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
  The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians. Recent studies have achieved promising results in automatic impression generation using large-scale medical text data. These models often require substantial amounts of medical text data and have poor generalization performance.
  arXiv  Detail & Related papers  (2023-04-17T17:13:42Z)
• Two heads are better than one: Enhancing medical representations by pre-training over structured and unstructured electronic health records [23.379185792773875]
  We propose a unified deep learning-based medical pre-trained language model, named UMM-PLM, to automatically learn representative features from multimodal EHRs. We first developed parallel unimodal information representation modules to capture the unimodal-specific characteristic, where unimodal representations were learned from each data source separately. A cross-modal module was further introduced to model the interactions between different modalities.
  arXiv  Detail & Related papers  (2022-01-25T06:14:49Z)
• A Meta-embedding-based Ensemble Approach for ICD Coding Prediction [64.42386426730695]
  International Classification of Diseases (ICD) are the de facto codes used globally for clinical coding. These codes enable healthcare providers to claim reimbursement and facilitate efficient storage and retrieval of diagnostic information. Our proposed approach enhances the performance of neural models by effectively training word vectors using routine medical data as well as external knowledge from scientific articles.
  arXiv  Detail & Related papers  (2021-02-26T17:49:58Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.