Nodule2vec: a 3D Deep Learning System for Pulmonary Nodule Retrieval Using Semantic Representation

• URL: http://arxiv.org/abs/2007.07081v1
• Date: Sat, 11 Jul 2020 16:26:16 GMT
• Title: Nodule2vec: a 3D Deep Learning System for Pulmonary Nodule Retrieval Using Semantic Representation
• Authors: Ilia Kravets, Tal Heletz, Hayit Greenspan
• Abstract summary: We present a deep learning system that transforms a 3D image of a pulmonary nodule from a CT scan into a low-dimensional embedding vector. We demonstrate that such a vector representation preserves semantic information about the nodule and offers a viable approach for content-based image retrieval (CBIR) A comparison between doctors and algorithm scores suggests that the benefit provided by the system to the radiologist end-user is comparable to obtaining a second radiologist's opinion.
• Score: 1.7403133838762446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Content-based retrieval supports a radiologist decision making process by presenting the doctor the most similar cases from the database containing both historical diagnosis and further disease development history. We present a deep learning system that transforms a 3D image of a pulmonary nodule from a CT scan into a low-dimensional embedding vector. We demonstrate that such a vector representation preserves semantic information about the nodule and offers a viable approach for content-based image retrieval (CBIR). We discuss the theoretical limitations of the available datasets and overcome them by applying transfer learning of the state-of-the-art lung nodule detection model. We evaluate the system using the LIDC-IDRI dataset of thoracic CT scans. We devise a similarity score and show that it can be utilized to measure similarity 1) between annotations of the same nodule by different radiologists and 2) between the query nodule and the top four CBIR results. A comparison between doctors and algorithm scores suggests that the benefit provided by the system to the radiologist end-user is comparable to obtaining a second radiologist's opinion.

Related papers

• 3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models [51.855377054763345]
  This paper introduces 3D-CT-GPT, a Visual Question Answering (VQA)-based medical visual language model for generating radiology reports from 3D CT scans. Experiments on both public and private datasets demonstrate that 3D-CT-GPT significantly outperforms existing methods in terms of report accuracy and quality.
  arXiv  Detail & Related papers  (2024-09-28T12:31:07Z)
• Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models [17.75505740079875]
  We explore the feasibility of leveraging language as a naturally high-quality supervision for chest CT imaging. We bootstrap the understanding of 3D chest CT images by distilling chest-related diagnostic knowledge from an extensively pre-trained 2D X-ray expert model. We train our model with over 12,000 pairs of chest CT images and radiology reports.
  arXiv  Detail & Related papers  (2024-04-07T12:17:40Z)
• UMedNeRF: Uncertainty-aware Single View Volumetric Rendering for Medical Neural Radiance Fields [38.62191342903111]
  We propose an Uncertainty-aware MedNeRF (UMedNeRF) network based on generated radiation fields. We show the results of CT projection rendering with a single X-ray and compare our method with other methods based on generated radiation fields.
  arXiv  Detail & Related papers  (2023-11-10T02:47:15Z)
• Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction like Radiologists [39.907916342786564]
  Lung cancer is a leading cause of death worldwide and early screening is critical for improving survival outcomes. In clinical practice, the contextual structure of nodules and the accumulated experience of radiologists are the two core elements related to the accuracy of identification of benign and malignant nodules. We propose a radiologist-inspired method to simulate the diagnostic process of radiologists, which is composed of context parsing and prototype recalling modules.
  arXiv  Detail & Related papers  (2023-07-20T12:38:17Z)
• Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling [66.75096111651062]
  We created a large-scale dataset of 10,021 thoracic CTs with 157 labels. We applied an ensemble of 3D anatomy segmentation models to extract anatomical pseudo-labels. Our resulting segmentation models demonstrated remarkable performance on CXR.
  arXiv  Detail & Related papers  (2023-06-06T18:01:08Z)
• Identification of lung nodules CT scan using YOLOv5 based on convolution neural network [0.0]
  This study was to identify the nodule that were developing in the lungs of the participants. One-stage detector YOLOv5 trained on 280 CT SCAN from a public dataset LIDC-IDRI based on segmented pulmonary nodules.
  arXiv  Detail & Related papers  (2022-12-31T17:31:22Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• Context-Aware Transformers For Spinal Cancer Detection and Radiological Grading [70.04389979779195]
  This paper proposes a novel transformer-based model architecture for medical imaging problems involving analysis of vertebrae. It considers two applications of such models in MR images: (a) detection of spinal metastases and the related conditions of vertebral fractures and metastatic cord compression. We show that by considering the context of vertebral bodies in the image, SCT improves the accuracy for several gradings compared to previously published model.
  arXiv  Detail & Related papers  (2022-06-27T10:31:03Z)
• Siamese Encoder-based Spatial-Temporal Mixer for Growth Trend Prediction of Lung Nodules on CT Scans [13.882367716329387]
  In the management of lung nodules, we are desirable to predict evolution in terms of its diameter variation on Computed Tomography (CT) scans. In order to improve the performance of growth trend prediction for lung nodules, it is vital to compare the changes of the same nodule in consecutive CT scans.
  arXiv  Detail & Related papers  (2022-06-07T06:44:56Z)
• CoRSAI: A System for Robust Interpretation of CT Scans of COVID-19 Patients Using Deep Learning [133.87426554801252]
  We adopted an approach based on using an ensemble of deep convolutionalneural networks for segmentation of lung CT scans. Using our models we are able to segment the lesions, evaluatepatients dynamics, estimate relative volume of lungs affected by lesions and evaluate the lung damage stage.
  arXiv  Detail & Related papers  (2021-05-25T12:06:55Z)
• XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors [118.27130593216096]
  A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane. To the best of our knowledge, this is the first work on radiograph view synthesis. We show that by gaining an understanding of radiography in 3D space, our method can be applied to radiograph bone extraction and suppression without groundtruth bone labels.
  arXiv  Detail & Related papers  (2020-12-04T05:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.