Graph Enhanced Contrastive Learning for Radiology Findings Summarization

• URL: http://arxiv.org/abs/2204.00203v1
• Date: Fri, 1 Apr 2022 04:39:44 GMT
• Title: Graph Enhanced Contrastive Learning for Radiology Findings Summarization
• Authors: Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, Tsung-Hui Chang
• Abstract summary: A section of a radiology report summarizes the most prominent observation from the findings. We propose a unified framework for exploiting both extra knowledge and the original findings. Key words and their relations can be extracted in an appropriate way to facilitate impression generation.
• Score: 25.377658879658306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The impression section of a radiology report summarizes the most prominent observation from the findings section and is the most important section for radiologists to communicate to physicians. Summarizing findings is time-consuming and can be prone to error for inexperienced radiologists, and thus automatic impression generation has attracted substantial attention. With the encoder-decoder framework, most previous studies explore incorporating extra knowledge (e.g., static pre-defined clinical ontologies or extra background information). Yet, they encode such knowledge by a separate encoder to treat it as an extra input to their models, which is limited in leveraging their relations with the original findings. To address the limitation, we propose a unified framework for exploiting both extra knowledge and the original findings in an integrated way so that the critical information (i.e., key words and their relations) can be extracted in an appropriate way to facilitate impression generation. In detail, for each input findings, it is encoded by a text encoder, and a graph is constructed through its entities and dependency tree. Then, a graph encoder (e.g., graph neural networks (GNNs)) is adopted to model relation information in the constructed graph. Finally, to emphasize the key words in the findings, contrastive learning is introduced to map positive samples (constructed by masking non-key words) closer and push apart negative ones (constructed by masking key words). The experimental results on OpenI and MIMIC-CXR confirm the effectiveness of our proposed method.

Related papers

• KiUT: Knowledge-injected U-Transformer for Radiology Report Generation [10.139767157037829]
  Radiology report generation aims to automatically generate a clinically accurate and coherent paragraph from the X-ray image. We propose a Knowledge-injected U-Transformer (KiUT) to learn multi-level visual representation and adaptively distill the information.
  arXiv  Detail & Related papers  (2023-06-20T07:27:28Z)
• Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [92.73584302508907]
  We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
  arXiv  Detail & Related papers  (2023-03-18T03:53:43Z)
• Improving Radiology Summarization with Radiograph and Anatomy Prompts [60.30659124918211]
  We propose a novel anatomy-enhanced multimodal model to promote impression generation. In detail, we first construct a set of rules to extract anatomies and put these prompts into each sentence to highlight anatomy characteristics. We utilize a contrastive learning module to align these two representations at the overall level and use a co-attention to fuse them at the sentence level.
  arXiv  Detail & Related papers  (2022-10-15T14:05:03Z)
• Cross-modal Memory Networks for Radiology Report Generation [30.13916304931662]
  Cross-modal memory networks (CMN) are proposed to enhance the encoder-decoder framework for radiology report generation. Our model is able to better align information from radiology images and texts so as to help generating more accurate reports in terms of clinical indicators.
  arXiv  Detail & Related papers  (2022-04-28T02:32:53Z)
• Word Graph Guided Summarization for Radiology Findings [24.790502861602075]
  We propose a novel method for automatic impression generation, where a word graph is constructed from the findings to record the critical words and their relations. A Word Graph guided Summarization model (WGSum) is designed to generate impressions with the help of the word graph. Experimental results on two datasets, OpenI and MIMIC-CXR, confirm the validity and effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2021-12-18T13:20:18Z)
• Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation [55.00308939833555]
  The PPKED includes three modules: Posterior Knowledge Explorer (PoKE), Prior Knowledge Explorer (PrKE) and Multi-domain Knowledge Distiller (MKD) PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias. PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias.
  arXiv  Detail & Related papers  (2021-06-13T11:10:02Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization [64.56399911605286]
  We propose SumGNN: knowledge summarization graph neural network, which is enabled by a subgraph extraction module. SumGNN outperforms the best baseline by up to 5.54%, and the performance gain is particularly significant in low data relation types.
  arXiv  Detail & Related papers  (2020-10-04T00:14:57Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.