ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans

• URL: http://arxiv.org/abs/2105.12810v1
• Date: Wed, 26 May 2021 20:00:31 GMT
• Title: ViPTT-Net: Video pretraining of spatio-temporal model for tuberculosis type classification from chest CT scans
• Authors: Hasib Zunair, Aimon Rahman, and Nabeel Mohammed
• Abstract summary: Pretraining has sparked groundswell of interest in deep learning to learn from limited data and improve generalization. We explore the idea of whether pretraining a model on realistic videos could improve performance rather than training the model from scratch. Our model termed as ViPTT-Net, was trained on over 1300 video clips with labels of human activities, and then fine-tuned on chest CT scans with labels of tuberculosis type.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pretraining has sparked groundswell of interest in deep learning workflows to learn from limited data and improve generalization. While this is common for 2D image classification tasks, its application to 3D medical imaging tasks like chest CT interpretation is limited. We explore the idea of whether pretraining a model on realistic videos could improve performance rather than training the model from scratch, intended for tuberculosis type classification from chest CT scans. To incorporate both spatial and temporal features, we develop a hybrid convolutional neural network (CNN) and recurrent neural network (RNN) model, where the features are extracted from each axial slice of the CT scan by a CNN, these sequence of image features are input to a RNN for classification of the CT scan. Our model termed as ViPTT-Net, was trained on over 1300 video clips with labels of human activities, and then fine-tuned on chest CT scans with labels of tuberculosis type. We find that pretraining the model on videos lead to better representations and significantly improved model validation performance from a kappa score of 0.17 to 0.35, especially for under-represented class samples. Our best method achieved 2nd place in the ImageCLEF 2021 Tuberculosis - TBT classification task with a kappa score of 0.20 on the final test set with only image information (without using clinical meta-data). All codes and models are made available.

Related papers

• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Strong Baseline and Bag of Tricks for COVID-19 Detection of CT Scans [2.696776905220987]
  Traditional deep learning frameworks encounter compatibility issues due to variations in slice numbers and resolutions in CT images. We propose a novel slice selection method for each CT dataset to address this limitation. In addition to the aforementioned methods, we explore various high-performance classification models, ultimately achieving promising results.
  arXiv  Detail & Related papers  (2023-03-15T09:52:28Z)
• CNN Filter Learning from Drawn Markers for the Detection of Suggestive Signs of COVID-19 in CT Images [58.720142291102135]
  We propose a method that does not require either large annotated datasets or backpropagation to estimate the filters of a convolutional neural network (CNN) For a few CT images, the user draws markers at representative normal and abnormal regions. The method generates a feature extractor composed of a sequence of convolutional layers, whose kernels are specialized in enhancing regions similar to the marked ones.
  arXiv  Detail & Related papers  (2021-11-16T15:03:42Z)
• Data Augmentation and CNN Classification For Automatic COVID-19 Diagnosis From CT-Scan Images On Small Dataset [0.0]
  We present an automatic COVID1-19 diagnosis framework from lung CT images. We propose a unique and effective data augmentation method using multiple Hounsfield Unit (HU) normalization windows. On the training/validation dataset, we achieve a patient classification accuracy of 93.39%.
  arXiv  Detail & Related papers  (2021-08-16T15:23:00Z)
• ViViT: A Video Vision Transformer [75.74690759089529]
  We present pure-transformer based models for video classification. Our model extracts-temporal tokens from the input video, which are then encoded by a series of transformer layers. We show how we can effectively regularise the model during training and leverage pretrained image models to be able to train on comparatively small datasets.
  arXiv  Detail & Related papers  (2021-03-29T15:27:17Z)
• COVID-19 identification from volumetric chest CT scans using a progressively resized 3D-CNN incorporating segmentation, augmentation, and class-rebalancing [4.446085353384894]
  COVID-19 is a global pandemic disease overgrowing worldwide. Computer-aided screening tools with greater sensitivity is imperative for disease diagnosis and prognosis. This article proposes a 3D Convolutional Neural Network (CNN)-based classification approach.
  arXiv  Detail & Related papers  (2021-02-11T18:16:18Z)
• Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans [72.04652116817238]
  We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification. We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
  arXiv  Detail & Related papers  (2021-01-14T03:45:01Z)
• A Multi-Stage Attentive Transfer Learning Framework for Improving COVID-19 Diagnosis [49.3704402041314]
  We propose a multi-stage attentive transfer learning framework for improving COVID-19 diagnosis. Our proposed framework consists of three stages to train accurate diagnosis models through learning knowledge from multiple source tasks and data of different domains. Importantly, we propose a novel self-supervised learning method to learn multi-scale representations for lung CT images.
  arXiv  Detail & Related papers  (2021-01-14T01:39:19Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)
• Improving Automated COVID-19 Grading with Convolutional Neural Networks in Computed Tomography Scans: An Ablation Study [3.072491397378425]
  This paper identifies a variety of components that increase the performance of CNN-based algorithms for COVID-19 grading from CT images. A 3D CNN with these components achieved an area under the ROC curve (AUC) of 0.934 on our test set of 105 CT scans and an AUC of 0.923 on a publicly available set of 742 CT scans.
  arXiv  Detail & Related papers  (2020-09-21T09:58:57Z)
• Accurate and Efficient Intracranial Hemorrhage Detection and Subtype Classification in 3D CT Scans with Convolutional and Long Short-Term Memory Neural Networks [20.4701676109641]
  We present our system for the RSNA Intracranial Hemorrhage Detection challenge. The proposed system is based on a lightweight deep neural network architecture composed of a convolutional neural network (CNN) We report a weighted mean log loss of 0.04989 on the final test set, which places us in the top 30 ranking (2%) from a total of 1345 participants.
  arXiv  Detail & Related papers  (2020-08-01T17:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.