Topicwise Separable Sentence Retrieval for Medical Report Generation

• URL: http://arxiv.org/abs/2405.04175v1
• Date: Tue, 7 May 2024 10:21:23 GMT
• Title: Topicwise Separable Sentence Retrieval for Medical Report Generation
• Authors: Junting Zhao, Yang Zhou, Zhihao Chen, Huazhu Fu, Liang Wan,
• Abstract summary: We introduce a Topicwise Separable Sentence Retrieval (Teaser) for medical report generation. To ensure comprehensive learning of both common and rare topics, we categorize queries into common and rare types, and then propose Topic Contrastive Loss. Experiments on the MIMIC-CXR and IU X-ray datasets demonstrate that Teaser surpasses state-of-the-art models.
• Score: 41.812337937025084
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated radiology reporting holds immense clinical potential in alleviating the burdensome workload of radiologists and mitigating diagnostic bias. Recently, retrieval-based report generation methods have garnered increasing attention due to their inherent advantages in terms of the quality and consistency of generated reports. However, due to the long-tail distribution of the training data, these models tend to learn frequently occurring sentences and topics, overlooking the rare topics. Regrettably, in many cases, the descriptions of rare topics often indicate critical findings that should be mentioned in the report. To address this problem, we introduce a Topicwise Separable Sentence Retrieval (Teaser) for medical report generation. To ensure comprehensive learning of both common and rare topics, we categorize queries into common and rare types to learn differentiated topics, and then propose Topic Contrastive Loss to effectively align topics and queries in the latent space. Moreover, we integrate an Abstractor module following the extraction of visual features, which aids the topic decoder in gaining a deeper understanding of the visual observational intent. Experiments on the MIMIC-CXR and IU X-ray datasets demonstrate that Teaser surpasses state-of-the-art models, while also validating its capability to effectively represent rare topics and establish more dependable correspondences between queries and topics.

Related papers

• Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning [65.54680361074882]
  Eye-gaze Guided Multi-modal Alignment (EGMA) framework harnesses eye-gaze data for better alignment of medical visual and textual features. We conduct downstream tasks of image classification and image-text retrieval on four medical datasets.
  arXiv  Detail & Related papers  (2024-03-19T03:59:14Z)
• Cross-Modal Causal Intervention for Medical Report Generation [109.83549148448469]
  Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance. Due to the spurious correlations within image-text data induced by visual and linguistic biases, it is challenging to generate accurate reports reliably describing lesion areas. We propose a novel Visual-Linguistic Causal Intervention (VLCI) framework for MRG, which consists of a visual deconfounding module (VDM) and a linguistic deconfounding module (LDM)
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Lesion Guided Explainable Few Weak-shot Medical Report Generation [25.15493013683396]
  We propose a lesion guided explainable few weak-shot medical report generation framework. It learns correlation between seen and novel classes through visual and semantic feature alignment. It aims to generate medical reports for diseases not observed in training.
  arXiv  Detail & Related papers  (2022-11-16T07:47:29Z)
• Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation [70.7778938191405]
  We propose a novel factored attention and embedding model (termed FAE-Gen) for the unstructured-view topic-related ultrasound report generation. The proposed FAE-Gen mainly consists of two modules, i.e., view-guided factored attention and topic-oriented factored embedding, which capture the homogeneous and heterogeneous morphological characteristic across different views.
  arXiv  Detail & Related papers  (2022-03-12T15:24:03Z)
• Extracting Radiological Findings With Normalized Anatomical Information Using a Span-Based BERT Relation Extraction Model [0.20999222360659603]
  Medical imaging reports distill the findings and observations of radiologists. Large-scale use of this text-encoded information requires converting the unstructured text to a structured, semantic representation. We explore the extraction and normalization of anatomical information in radiology reports that is associated with radiological findings.
  arXiv  Detail & Related papers  (2021-08-20T15:02:59Z)
• Variational Topic Inference for Chest X-Ray Report Generation [102.04931207504173]
  Report generation for medical imaging promises to reduce workload and assist diagnosis in clinical practice. Recent work has shown that deep learning models can successfully caption natural images. We propose variational topic inference for automatic report generation.
  arXiv  Detail & Related papers  (2021-07-15T13:34:38Z)
• Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition [142.42920413017163]
  Current methods often generate the most common sentences due to dataset bias for individual case. We propose a novel framework that unifies template retrieval and sentence generation to handle both common and rare abnormality.
  arXiv  Detail & Related papers  (2021-01-09T04:33:27Z)
• Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays [6.686095511538683]
  This work focuses on reporting abnormal findings on radiology images. We propose a method to identify abnormal findings from the reports in addition to grouping them with unsupervised clustering and minimal rules. We demonstrate that our method is able to retrieve abnormal findings and outperforms existing generation models on both clinical correctness and text generation metrics.
  arXiv  Detail & Related papers  (2020-10-06T04:18:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.