ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning

• URL: http://arxiv.org/abs/2306.06466v1
• Date: Sat, 10 Jun 2023 15:36:04 GMT
• Title: ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning
• Authors: Wenjun Hou, Kaishuai Xu, Yi Cheng, Wenjie Li, Jiang Liu
• Abstract summary: We propose an observation-guided radiology report generation framework (ORGAN) It first produces an observation plan and then feeds both the plan and radiographs for report generation. Our framework outperforms previous state-of-the-art methods regarding text quality and clinical efficacy.
• Score: 9.316999438459794
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores the task of radiology report generation, which aims at generating free-text descriptions for a set of radiographs. One significant challenge of this task is how to correctly maintain the consistency between the images and the lengthy report. Previous research explored solving this issue through planning-based methods, which generate reports only based on high-level plans. However, these plans usually only contain the major observations from the radiographs (e.g., lung opacity), lacking much necessary information, such as the observation characteristics and preliminary clinical diagnoses. To address this problem, the system should also take the image information into account together with the textual plan and perform stronger reasoning during the generation process. In this paper, we propose an observation-guided radiology report generation framework (ORGAN). It first produces an observation plan and then feeds both the plan and radiographs for report generation, where an observation graph and a tree reasoning mechanism are adopted to precisely enrich the plan information by capturing the multi-formats of each observation. Experimental results demonstrate that our framework outperforms previous state-of-the-art methods regarding text quality and clinical efficacy

Related papers

• Medical Report Generation Is A Multi-label Classification Problem [38.64929236412092]
  We propose rethinking medical report generation as a multi-label classification problem. We introduce a novel report generation framework based on BLIP integrated with classified key nodes. Our experiments demonstrate that leveraging key nodes can achieve state-of-the-art (SOTA) performance, surpassing existing approaches across two benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-30T20:43:35Z)
• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Reading Radiology Imaging Like The Radiologist [3.218449686637963]
  We design a factual consistency captioning generator to generate more accurate and factually consistent disease descriptions. Our framework can find most similar reports for a given disease from the CXR database by retrieving a disease-oriented mask.
  arXiv  Detail & Related papers  (2023-07-12T05:36:47Z)
• Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation [92.73584302508907]
  We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning. In detail, the fundamental structure of our graph is pre-constructed from general knowledge. Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
  arXiv  Detail & Related papers  (2023-03-18T03:53:43Z)
• A Self-Guided Framework for Radiology Report Generation [10.573538773141715]
  A self-guided framework (SGF) is developed to generate medical reports with annotated disease labels. SGF uses unsupervised and supervised deep learning methods to mimic the process of human learning and writing. Our results highlight the capacity of the proposed framework to distinguish fined-grained visual details between words.
  arXiv  Detail & Related papers  (2022-06-19T11:09:27Z)
• Variational Topic Inference for Chest X-Ray Report Generation [102.04931207504173]
  Report generation for medical imaging promises to reduce workload and assist diagnosis in clinical practice. Recent work has shown that deep learning models can successfully caption natural images. We propose variational topic inference for automatic report generation.
  arXiv  Detail & Related papers  (2021-07-15T13:34:38Z)
• Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation [55.00308939833555]
  The PPKED includes three modules: Posterior Knowledge Explorer (PoKE), Prior Knowledge Explorer (PrKE) and Multi-domain Knowledge Distiller (MKD) PoKE explores the posterior knowledge, which provides explicit abnormal visual regions to alleviate visual data bias. PrKE explores the prior knowledge from the prior medical knowledge graph (medical knowledge) and prior radiology reports (working experience) to alleviate textual data bias.
  arXiv  Detail & Related papers  (2021-06-13T11:10:02Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)
• Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reports [5.6070625920019825]
  Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. The complex structures between and within sections of the reports pose a great challenge to the automatic report generation. We propose a novel framework that exploits the structure information between and within report sections for generating CXR imaging reports.
  arXiv  Detail & Related papers  (2020-04-26T02:29:20Z)
• When Radiology Report Generation Meets Knowledge Graph [17.59749125131158]
  The accuracy of positive disease keyword mentions is critical in radiology image reporting. The evaluation of reporting quality should focus more on matching the disease keywords and their associated attributes. We propose a new evaluation metric for radiology image reporting with the assistance of the same composed graph.
  arXiv  Detail & Related papers  (2020-02-19T16:39:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.