Can Large Language Models abstract Medical Coded Language?

• URL: http://arxiv.org/abs/2403.10822v3
• Date: Thu, 6 Jun 2024 21:58:49 GMT
• Title: Can Large Language Models abstract Medical Coded Language?
• Authors: Simon A. Lee, Timothy Lindsey,
• Abstract summary: Large language models (LLMs) are aware of medical code and can accurately generate names from these codes. This study evaluates whether large language models (LLMs) are aware of medical code and can accurately generate names from these codes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have become a pivotal research area, potentially making beneficial contributions in fields like healthcare where they can streamline automated billing and decision support. However, the frequent use of specialized coded languages like ICD-10, which are regularly updated and deviate from natural language formats, presents potential challenges for LLMs in creating accurate and meaningful latent representations. This raises concerns among healthcare professionals about potential inaccuracies or ``hallucinations" that could result in the direct impact of a patient. Therefore, this study evaluates whether large language models (LLMs) are aware of medical code ontologies and can accurately generate names from these codes. We assess the capabilities and limitations of both general and biomedical-specific generative models, such as GPT, LLaMA-2, and Meditron, focusing on their proficiency with domain-specific terminologies. While the results indicate that LLMs struggle with coded language, we offer insights on how to adapt these models to reason more effectively.

Related papers

• Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Grounding [9.144030136201476]
  Multimodal large language models (MLLMs) inherit the superior text understanding capabilities of LLMs and extend these capabilities to multimodal scenarios. These models achieve excellent results in the general domain of multimodal tasks. However, in the medical domain, the substantial training costs and the requirement for extensive medical data pose challenges to the development of medical MLLMs.
  arXiv  Detail & Related papers  (2024-10-31T11:07:26Z)
• The Role of Language Models in Modern Healthcare: A Comprehensive Review [2.048226951354646]
  The application of large language models (LLMs) in healthcare has gained significant attention. This review examines the trajectory of language models from their early stages to the current state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-09-25T12:15:15Z)
• Retrieve, Generate, Evaluate: A Case Study for Medical Paraphrases Generation with Small Language Models [2.4851820343103035]
  We introduce pRAGe, a pipeline for Retrieval Augmented Generation and evaluation of medical paraphrases generation using Small Language Models (SLM) We study the effectiveness of SLMs and the impact of external knowledge base for medical paraphrase generation in French.
  arXiv  Detail & Related papers  (2024-07-23T15:17:11Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora. But can these models relate corresponding concepts across languages, effectively being crosslingual? This study evaluates six state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• Towards Building Multilingual Language Model for Medicine [54.1382395897071]
  We construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages. We propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks.
  arXiv  Detail & Related papers  (2024-02-21T17:47:20Z)
• EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge [28.409333447902693]
  Large language models (LLMs) achieve remarkable performance in comprehensive and generative ability. In this work, we focus on the particular disease of Epilepsy with Japanese language and introduce a customized LLM termed as EpilepsyLLM. The datasets contain knowledge of basic information about disease, common treatment methods and drugs, and important notes in life and work.
  arXiv  Detail & Related papers  (2024-01-11T13:39:00Z)
• Large language models in healthcare and medical domain: A review [4.456243157307507]
  Large language models (LLMs) provide proficient responses to free-text queries. This review explores the potential of LLMs to amplify the efficiency and effectiveness of diverse healthcare applications.
  arXiv  Detail & Related papers  (2023-12-12T20:54:51Z)
• L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models [102.00201523306986]
  We present L2CEval, a systematic evaluation of the language-to-code generation capabilities of large language models (LLMs) We analyze the factors that potentially affect their performance, such as model size, pretraining data, instruction tuning, and different prompting methods. In addition to assessing model performance, we measure confidence calibration for the models and conduct human evaluations of the output programs.
  arXiv  Detail & Related papers  (2023-09-29T17:57:00Z)
• Evaluating Large Language Models for Radiology Natural Language Processing [68.98847776913381]
  The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP) This study seeks to bridge this gap by critically evaluating thirty two LLMs in interpreting radiology reports.
  arXiv  Detail & Related papers  (2023-07-25T17:57:18Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• Can large language models build causal graphs? [54.74910640970968]
  Large language models (LLMs) represent an opportunity to ease the process of building causal graphs. LLMs have been shown to be brittle to the choice of probing words, context, and prompts that the user employs.
  arXiv  Detail & Related papers  (2023-03-07T22:05:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.