Automated Report Generation for Lung Cytological Images Using a CNN Vision Classifier and Multiple-Transformer Text Decoders: Preliminary Study

• URL: http://arxiv.org/abs/2403.18151v1
• Date: Tue, 26 Mar 2024 23:32:29 GMT
• Title: Automated Report Generation for Lung Cytological Images Using a CNN Vision Classifier and Multiple-Transformer Text Decoders: Preliminary Study
• Authors: Atsushi Teramoto, Ayano Michiba, Yuka Kiriyama, Tetsuya Tsukamoto, Kazuyoshi Imaizumi, Hiroshi Fujita,
• Abstract summary: The sensitivity and specificity were 100% and 96.4%, respectively, for automated benign and malignant case classification. The grammar and style of the generated texts were confirmed as correct and in better agreement with gold standard.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cytology plays a crucial role in lung cancer diagnosis. Pulmonary cytology involves cell morphological characterization in the specimen and reporting the corresponding findings, which are extremely burdensome tasks. In this study, we propose a report-generation technique for lung cytology images. In total, 71 benign and 135 malignant pulmonary cytology specimens were collected. Patch images were extracted from the captured specimen images, and the findings were assigned to each image as a dataset for report generation. The proposed method consists of a vision model and a text decoder. In the former, a convolutional neural network (CNN) is used to classify a given image as benign or malignant, and the features related to the image are extracted from the intermediate layer. Independent text decoders for benign and malignant cells are prepared for text generation, and the text decoder switches according to the CNN classification results. The text decoder is configured using a Transformer that uses the features obtained from the CNN for report generation. Based on the evaluation results, the sensitivity and specificity were 100% and 96.4%, respectively, for automated benign and malignant case classification, and the saliency map indicated characteristic benign and malignant areas. The grammar and style of the generated texts were confirmed as correct and in better agreement with gold standard compared to existing LLM-based image-captioning methods and single-text-decoder ablation model. These results indicate that the proposed method is useful for pulmonary cytology classification and reporting.

Related papers

• COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images [3.5418498524791766]
  This research is development of a novel counterfactual inpainting approach (COIN) COIN flips the predicted classification label from abnormal to normal by using a generative model. The effectiveness of the method is demonstrated by segmenting synthetic targets and actual kidney tumors from CT images acquired from Tartu University Hospital in Estonia.
  arXiv  Detail & Related papers  (2024-04-19T12:09:49Z)
• Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERT [1.0819408603463427]
  We show that using an existing pre-trained Vision Transformer (ViT) to encode 4096x4096 sized patches of the Whole Slide Image (WSI) and a pre-trained Bidirectional Representations from Transformers (BERT) model for report generation, we can build a performant and portable report generation mechanism. Our method allows us to not only generate and evaluate captions that describe the image, but also helps us classify the image into tissue types and the gender of the patient as well.
  arXiv  Detail & Related papers  (2023-12-03T15:56:09Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting [13.151444796296868]
  We introduce a novel adaptation of Faster R-CNN in which finding detection is performed for the candidate bounding boxes extracted during anatomical structure localisation. We use the resulting bounding box feature representations as our set of finding-aware anatomical tokens. We show that task-aware anatomical tokens give state-of-the-art performance when integrated into an automated reporting pipeline.
  arXiv  Detail & Related papers  (2023-08-30T11:35:21Z)
• Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification [58.147396879490124]
  Our few-shot generation method, named XM-GAN, takes one base and a pair of reference tissue images as input and generates high-quality yet diverse images. To the best of our knowledge, we are the first to investigate few-shot generation in colorectal tissue images.
  arXiv  Detail & Related papers  (2023-04-04T17:50:30Z)
• Texture Characterization of Histopathologic Images Using Ecological Diversity Measures and Discrete Wavelet Transform [82.53597363161228]
  This paper proposes a method for characterizing texture across histopathologic images with a considerable success rate. It is possible to quantify the intrinsic properties of such images with promising accuracy on two HI datasets.
  arXiv  Detail & Related papers  (2022-02-27T02:19:09Z)
• Deep Learning models for benign and malign Ocular Tumor Growth Estimation [3.1558405181807574]
  Clinicians often face issues in selecting suitable image processing algorithm for medical imaging data. A strategy for the selection of a proper model is presented here.
  arXiv  Detail & Related papers  (2021-07-09T05:40:25Z)
• Wide & Deep neural network model for patch aggregation in CNN-based prostate cancer detection systems [51.19354417900591]
  Prostate cancer (PCa) is one of the leading causes of death among men, with almost 1.41 million new cases and around 375,000 deaths in 2020. To perform an automatic diagnosis, prostate tissue samples are first digitized into gigapixel-resolution whole-slide images. Small subimages called patches are extracted and predicted, obtaining a patch-level classification.
  arXiv  Detail & Related papers  (2021-05-20T18:13:58Z)
• Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation [107.3538598876467]
  We propose an Auxiliary Signal-Guided Knowledge-Decoder (ASGK) to mimic radiologists' working patterns. ASGK integrates internal visual feature fusion and external medical linguistic information to guide medical knowledge transfer and learning.
  arXiv  Detail & Related papers  (2020-06-06T01:00:15Z)
• Representation Learning of Histopathology Images using Graph Neural Networks [12.427740549056288]
  We propose a two-stage framework for WSI representation learning. We sample relevant patches using a color-based method and use graph neural networks to learn relations among sampled patches to aggregate the image information into a single vector representation. We demonstrate the performance of our approach for discriminating two sub-types of lung cancers, Lung Adenocarcinoma (LUAD) & Lung Squamous Cell Carcinoma (LUSC)
  arXiv  Detail & Related papers  (2020-04-16T00:09:20Z)
• Understanding the robustness of deep neural network classifiers for breast cancer screening [52.50078591615855]
  Deep neural networks (DNNs) show promise in breast cancer screening, but their robustness to input perturbations must be better understood before they can be clinically implemented. We measure the sensitivity of a radiologist-level screening mammogram image classifier to four commonly studied input perturbations. We also perform a detailed analysis on the effects of low-pass filtering, and find that it degrades the visibility of clinically meaningful features.
  arXiv  Detail & Related papers  (2020-03-23T01:26:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.