Designing an Improved Deep Learning-based Model for COVID-19 Recognition in Chest X-ray Images: A Knowledge Distillation Approach

• URL: http://arxiv.org/abs/2301.02735v2
• Date: Mon, 4 Sep 2023 12:42:20 GMT
• Title: Designing an Improved Deep Learning-based Model for COVID-19 Recognition in Chest X-ray Images: A Knowledge Distillation Approach
• Authors: AmirReza BabaAhmadi, Sahar Khalafi, Masoud ShariatPanahi, Moosa Ayati
• Abstract summary: This study uses two neural networks to improve feature extraction from our dataset: VGG19 and ResNet50V2. MobileNetV2 excels at extracting semantic features while requiring minimal computation on mobile and embedded devices.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: COVID-19 has adversely affected humans and societies in different aspects. Numerous people have perished due to inaccurate COVID-19 identification and, consequently, a lack of appropriate medical treatment. Numerous solutions based on manual and automatic feature extraction techniques have been investigated to address this issue by researchers worldwide. Typically, automatic feature extraction methods, particularly deep learning models, necessitate a powerful hardware system to perform the necessary computations. Unfortunately, many institutions and societies cannot benefit from these advancements due to the prohibitively high cost of high-quality hardware equipment. As a result, this study focused on two primary goals: first, lowering the computational costs associated with running the proposed model on embedded devices, mobile devices, and conventional computers; and second, improving the model's performance in comparison to previously published methods (at least performs on par with state-of-the-art models) in order to ensure its performance and accuracy for the medical recognition task. This study used two neural networks to improve feature extraction from our dataset: VGG19 and ResNet50V2. Both of these networks are capable of providing semantic features from the nominated dataset. To this end, An alternative network was considered, namely MobileNetV2, which excels at extracting semantic features while requiring minimal computation on mobile and embedded devices. Knowledge distillation (KD) was used to transfer knowledge from the teacher network (concatenated ResNet50V2 and VGG19) to the student network (MobileNetV2) to improve MobileNetV2 performance and to achieve a robust and accurate model for the COVID-19 identification task from chest X-ray images.

Related papers

• Masked LoGoNet: Fast and Accurate 3D Image Analysis for Medical Domain [48.440691680864745]
  We introduce a new neural network architecture, termed LoGoNet, with a tailored self-supervised learning (SSL) method. LoGoNet integrates a novel feature extractor within a U-shaped architecture, leveraging Large Kernel Attention (LKA) and a dual encoding strategy. We propose a novel SSL method tailored for 3D images to compensate for the lack of large labeled datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:06:58Z)
• Quantization-aware Neural Architectural Search for Intrusion Detection [5.010685611319813]
  We present a design methodology that automatically trains and evolves quantized neural network (NN) models that are a thousand times smaller than state-of-the-art NNs. The number of LUTs utilized by this network when deployed to an FPGA is between 2.3x and 8.5x smaller with performance comparable to prior work.
  arXiv  Detail & Related papers  (2023-11-07T18:35:29Z)
• Defect Classification in Additive Manufacturing Using CNN-Based Vision Processing [76.72662577101988]
  This paper examines two scenarios: first, using convolutional neural networks (CNNs) to accurately classify defects in an image dataset from AM and second, applying active learning techniques to the developed classification model. This allows the construction of a human-in-the-loop mechanism to reduce the size of the data required to train and generate training data.
  arXiv  Detail & Related papers  (2023-07-14T14:36:58Z)
• Benchmarking Deep Learning Frameworks for Automated Diagnosis of Ocular Toxoplasmosis: A Comprehensive Approach to Classification and Segmentation [1.3701366534590498]
  Ocular Toxoplasmosis (OT) is a common eye infection caused by T. gondii that can cause vision problems. This research seeks to provide a guide for future researchers looking to utilise DL techniques and develop a cheap, automated, easy-to-use, and accurate diagnostic method.
  arXiv  Detail & Related papers  (2023-05-18T13:42:15Z)
• A Light-weight Deep Learning Model for Remote Sensing Image Classification [70.66164876551674]
  We present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC) By conducting extensive experiments on the NWPU-RESISC45 benchmark, our proposed teacher-student models outperforms the state-of-the-art systems.
  arXiv  Detail & Related papers  (2023-02-25T09:02:01Z)
• Backdoor Attack Detection in Computer Vision by Applying Matrix Factorization on the Weights of Deep Networks [6.44397009982949]
  We introduce a novel method for backdoor detection that extracts features from pre-trained DNN's weights. In comparison to other detection techniques, this has a number of benefits, such as not requiring any training data. Our method outperforms the competing algorithms in terms of efficiency and is more accurate, helping to ensure the safe application of deep learning and AI.
  arXiv  Detail & Related papers  (2022-12-15T20:20:18Z)
• On-Device Domain Generalization [93.79736882489982]
  Domain generalization is critical to on-device machine learning applications. We find that knowledge distillation is a strong candidate for solving the problem. We propose a simple idea called out-of-distribution knowledge distillation (OKD), which aims to teach the student how the teacher handles (synthetic) out-of-distribution data.
  arXiv  Detail & Related papers  (2022-09-15T17:59:31Z)
• Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition [12.682984063354748]
  This study introduces an end-to-end Vision-to-Sensor Knowledge Distillation (VSKD) framework. In this VSKD framework, only time-series data, i.e., accelerometer data, is needed from wearable devices during the testing phase. This framework will not only reduce the computational demands on edge devices, but also produce a learning model that closely matches the performance of the computational expensive multi-modal approach.
  arXiv  Detail & Related papers  (2021-10-08T15:06:38Z)
• ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search [6.316693022958222]
  Deep neural networks have been outperforming conventional machine learning algorithms in many computer vision-related tasks. The majority of devices are harnessing the cloud computing methodology in which outstanding deep learning models are responsible for analyzing the data on the server. In this paper, a new framework for deploying on IoT devices has been proposed which can take advantage of both the cloud and the on-device models.
  arXiv  Detail & Related papers  (2021-06-21T04:47:53Z)
• Making DensePose fast and light [78.49552144907513]
  Existing neural network models capable of solving this task are heavily parameterized. To enable Dense Pose inference on the end device with current models, one needs to support an expensive server-side infrastructure and have a stable internet connection. In this work, we target the problem of redesigning the DensePose R-CNN model's architecture so that the final network retains most of its accuracy but becomes more light-weight and fast.
  arXiv  Detail & Related papers  (2020-06-26T19:42:20Z)
• A Data and Compute Efficient Design for Limited-Resources Deep Learning [68.55415606184]
  equivariant neural networks have gained increased interest in the deep learning community. They have been successfully applied in the medical domain where symmetries in the data can be effectively exploited to build more accurate and robust models. Mobile, on-device implementations of deep learning solutions have been developed for medical applications. However, equivariant models are commonly implemented using large and computationally expensive architectures, not suitable to run on mobile devices. In this work, we design and test an equivariant version of MobileNetV2 and further optimize it with model quantization to enable more efficient inference.
  arXiv  Detail & Related papers  (2020-04-21T00:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.