Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning

• URL: http://arxiv.org/abs/2108.05067v1
• Date: Wed, 11 Aug 2021 07:12:57 GMT
• Title: Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning
• Authors: Guangyi Liu, Yinghong Liao, Fuyu Wang, Bin Zhang, Lu Zhang, Xiaodan Liang, Xiang Wan, Shaolin Li, Zhen Li, Shuixing Zhang, Shuguang Cui
• Abstract summary: We propose to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans. This model adopts an alternate learning strategy with two procedures that are knowledge pretraining and transferring. For automatic medical report generation on the COVID-19 cases, we constructed a dataset of 368 medical findings in Chinese and 1104 chest CT scans.
• Score: 70.71564065885542
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical imaging technologies, including computed tomography (CT) or chest X-Ray (CXR), are largely employed to facilitate the diagnosis of the COVID-19. Since manual report writing is usually too time-consuming, a more intelligent auxiliary medical system that could generate medical reports automatically and immediately is urgently needed. In this article, we propose to use the medical visual language BERT (Medical-VLBERT) model to identify the abnormality on the COVID-19 scans and generate the medical report automatically based on the detected lesion regions. To produce more accurate medical reports and minimize the visual-and-linguistic differences, this model adopts an alternate learning strategy with two procedures that are knowledge pretraining and transferring. To be more precise, the knowledge pretraining procedure is to memorize the knowledge from medical texts, while the transferring procedure is to utilize the acquired knowledge for professional medical sentences generations through observations of medical images. In practice, for automatic medical report generation on the COVID-19 cases, we constructed a dataset of 368 medical findings in Chinese and 1104 chest CT scans from The First Affiliated Hospital of Jinan University, Guangzhou, China, and The Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, China. Besides, to alleviate the insufficiency of the COVID-19 training samples, our model was first trained on the large-scale Chinese CX-CHR dataset and then transferred to the COVID-19 CT dataset for further fine-tuning. The experimental results showed that Medical-VLBERT achieved state-of-the-art performances on terminology prediction and report generation with the Chinese COVID-19 CT dataset and the CX-CHR dataset. The Chinese COVID-19 CT dataset is available at https://covid19ct.github.io/.

Related papers

• MEDNC: Multi-ensemble deep neural network for COVID-19 diagnosis [29.909378035039214]
  We propose the deep learning framework MEDNC for automatic prediction and diagnosis of COVID-19 using computed tomography (CT) images. Our model was trained using two publicly available sets of COVID-19 data. Results indicated that the MEDNC greatly enhanced the detection of COVID-19 infections, reaching an accuracy of 98.79% and 99.82% respectively.
  arXiv  Detail & Related papers  (2023-04-25T20:26:05Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• When Accuracy Meets Privacy: Two-Stage Federated Transfer Learning Framework in Classification of Medical Images on Limited Data: A COVID-19 Case Study [77.34726150561087]
  COVID-19 pandemic has spread rapidly and caused a shortage of global medical resources. CNN has been widely utilized and verified in analyzing medical images.
  arXiv  Detail & Related papers  (2022-03-24T02:09:41Z)
• Classification of COVID-19 on chest X-Ray images using Deep Learning model with Histogram Equalization and Lungs Segmentation [1.6019444314820142]
  We present our study based on deep learning architecture for detecting covid-19 infected lungs using chest X-rays. Our novel approach combining well-known pre-processing techniques, feature extraction methods, and dataset balancing method, lead us to an outstanding rate of recognition of 98%.
  arXiv  Detail & Related papers  (2021-12-05T05:04:38Z)
• Learning from Pseudo Lesion: A Self-supervised Framework for COVID-19 Diagnosis [22.54540093657541]
  The Coronavirus disease 2019 (COVID-19) has rapidly spread all over the world since its first report in December 2019. In recent years, deep learning-based approaches have shown impressive performance in myriad image recognition tasks. We proposed in this paper a novel self-supervised pretraining method based on pseudo lesions generation and restoration for COVID-19 diagnosis.
  arXiv  Detail & Related papers  (2021-06-23T11:21:30Z)
• In-Line Image Transformations for Imbalanced, Multiclass Computer Vision Classification of Lung Chest X-Rays [91.3755431537592]
  This study aims to leverage a body of literature in order to apply image transformations that would serve to balance the lack of COVID-19 LCXR data. Deep learning techniques such as convolutional neural networks (CNNs) are able to select features that distinguish between healthy and disease states. This study utilizes a simple CNN architecture for high-performance multiclass LCXR classification at 94 percent accuracy.
  arXiv  Detail & Related papers  (2021-04-06T02:01:43Z)
• Few-shot Learning for CT Scan based COVID-19 Diagnosis [33.26861533338019]
  Coronavirus disease 2019 (COVID-19) is a Public Health Emergency of International Concern infecting more than 40 million people across 188 countries and territories. Deep learning approaches have become an effective tool for automatic screening of medical images, and it is also being considered for COVID-19 diagnosis. We propose a supervised domain adaption based COVID-19 CT diagnostic method which can perform effectively when only a small samples of labeled CT scans are available.
  arXiv  Detail & Related papers  (2021-02-01T02:37:49Z)
• COVIDNet-CT: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest CT Images [75.74756992992147]
  We introduce COVIDNet-CT, a deep convolutional neural network architecture that is tailored for detection of COVID-19 cases from chest CT images. We also introduce COVIDx-CT, a benchmark CT image dataset derived from CT imaging data collected by the China National Center for Bioinformation.
  arXiv  Detail & Related papers  (2020-09-08T15:49:55Z)
• COVID-19-CT-CXR: a freely accessible and weakly labeled chest X-ray and CT image collection on COVID-19 from biomedical literature [19.00121006721942]
  We present COVID-19-CT-CXR, a public database of COVID-19 CXR and CT images automatically extracted from COVID-19-relevant articles. The final database includes 1,327 CT and 263 CXR images with their relevant text.
  arXiv  Detail & Related papers  (2020-06-11T04:00:56Z)
• Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation and Diagnosis for COVID-19 [71.41929762209328]
  The pandemic of coronavirus disease 2019 (COVID-19) is spreading all over the world. Medical imaging such as X-ray and computed tomography (CT) plays an essential role in the global fight against COVID-19. The recently emerging artificial intelligence (AI) technologies further strengthen the power of the imaging tools and help medical specialists.
  arXiv  Detail & Related papers  (2020-04-06T15:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.