CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

• URL: http://arxiv.org/abs/2406.11451v3
• Date: Wed, 18 Sep 2024 06:53:40 GMT
• Title: CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation
• Authors: Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, Lihua Zhang,
• Abstract summary: We propose a chain-of-medical-thought approach (CoMT) to mitigate hallucinations in medical report generation. CoMT intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures.
• Score: 20.59298361626719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. All resources of this work will be released soon.

Related papers

• MINDSETS: Multi-omics Integration with Neuroimaging for Dementia Subtyping and Effective Temporal Study [0.7751705157998379]
  Alzheimer's disease (AD) and vascular dementia (VaD) are the two most prevalent dementia types. This paper presents an innovative multi-omics approach to accurately differentiate AD from VaD, achieving a diagnostic accuracy of 89.25%.
  arXiv  Detail & Related papers  (2024-11-06T10:13:28Z)
• MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models [49.765466293296186]
  Recent progress in Medical Large Vision-Language Models (Med-LVLMs) has opened up new possibilities for interactive diagnostic tools. Med-LVLMs often suffer from factual hallucination, which can lead to incorrect diagnoses. We propose a versatile multimodal RAG system, MMed-RAG, designed to enhance the factuality of Med-LVLMs.
  arXiv  Detail & Related papers  (2024-10-16T23:03:27Z)
• Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports [51.45762396192655]
  Multimodal large language models (MLLMs) have recently transformed many domains, significantly affecting the medical field. Notably, Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models have epitomized a paradigm shift in Artificial General Intelligence for computer vision. This study evaluated the performance of the Gemini, GPT-4, and 4 popular large models for an exhaustive evaluation across 14 medical imaging datasets.
  arXiv  Detail & Related papers  (2024-07-08T09:08:42Z)
• MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis [28.421857904824627]
  MiniGPT-Med is a vision-language model derived from large-scale language models and tailored for medical applications. It is capable of performing tasks such as medical report generation, visual question answering (VQA), and disease identification within medical imagery. It achieves state-of-the-art performance on medical report generation, higher than the previous best model by 19% accuracy.
  arXiv  Detail & Related papers  (2024-07-04T18:21:10Z)
• Towards Knowledge-Infused Automated Disease Diagnosis Assistant [14.150224660741939]
  We build a diagnosis assistant to assist doctors, which identifies diseases based on patient-doctor interaction. We propose a two-channel, discourse-aware disease diagnosis model (KI-DDI), where the first channel encodes patient-doctor communication. In the next stage, the conversation and knowledge graph embeddings are infused together and fed to a deep neural network for disease identification.
  arXiv  Detail & Related papers  (2024-05-18T05:18:50Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Leveraging A Medical Knowledge Graph into Large Language Models for Diagnosis Prediction [7.5569033426158585]
  We propose an innovative approach for augmenting the proficiency of Large Language Models (LLMs) in automated diagnosis generation. We derive the KG from the National Library of Medicine's Unified Medical Language System (UMLS), a robust repository of biomedical knowledge. Our approach offers an explainable diagnostic pathway, edging us closer to the realization of AI-augmented diagnostic decision support systems.
  arXiv  Detail & Related papers  (2023-08-28T06:05:18Z)
• PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation [7.508437260320598]
  We propose diagnosis-driven prompts for medical report generation (PromptMRG) PromptMRG is based on encoder-decoder architecture with an extra disease classification branch. Cross-modal feature enhancement retrieves similar reports from the database to assist the diagnosis of a query image.
  arXiv  Detail & Related papers  (2023-08-24T07:10:31Z)
• Cross-Modal Causal Intervention for Medical Report Generation [109.83549148448469]
  Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance. Due to the spurious correlations within image-text data induced by visual and linguistic biases, it is challenging to generate accurate reports reliably describing lesion areas. We propose a novel Visual-Linguistic Causal Intervention (VLCI) framework for MRG, which consists of a visual deconfounding module (VDM) and a linguistic deconfounding module (LDM)
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Inheritance-guided Hierarchical Assignment for Clinical Automatic Diagnosis [50.15205065710629]
  Clinical diagnosis, which aims to assign diagnosis codes for a patient based on the clinical note, plays an essential role in clinical decision-making. We propose a novel framework to combine the inheritance-guided hierarchical assignment and co-occurrence graph propagation for clinical automatic diagnosis.
  arXiv  Detail & Related papers  (2021-01-27T13:16:51Z)
• Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation [150.52617238140868]
  We propose low-resource medical dialogue generation to transfer the diagnostic experience from source diseases to target ones. We also develop a Graph-Evolving Meta-Learning framework that learns to evolve the commonsense graph for reasoning disease-symptom correlations in a new disease.
  arXiv  Detail & Related papers  (2020-12-22T13:20:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.