Efficient Medical Image Segmentation Based on Knowledge Distillation

• URL: http://arxiv.org/abs/2108.09987v1
• Date: Mon, 23 Aug 2021 07:41:10 GMT
• Title: Efficient Medical Image Segmentation Based on Knowledge Distillation
• Authors: Dian Qin, Jiajun Bu, Zhe Liu, Xin Shen, Sheng Zhou, Jingjun Gu, Zhijua Wang, Lei Wu, Huifen Dai
• Abstract summary: We propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network. We also devise a novel distillation module tailored for medical image segmentation to transfer semantic region information from teacher to student network. We demonstrate that a lightweight network distilled by our method has non-negligible value in the scenario which requires relatively high operating speed and low storage usage.
• Score: 30.857487609003197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances have been made in applying convolutional neural networks to achieve more precise prediction results for medical image segmentation problems. However, the success of existing methods has highly relied on huge computational complexity and massive storage, which is impractical in the real-world scenario. To deal with this problem, we propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network. This architecture empowers the lightweight network to get a significant improvement on segmentation capability while retaining its runtime efficiency. We further devise a novel distillation module tailored for medical image segmentation to transfer semantic region information from teacher to student network. It forces the student network to mimic the extent of difference of representations calculated from different tissue regions. This module avoids the ambiguous boundary problem encountered when dealing with medical imaging but instead encodes the internal information of each semantic region for transferring. Benefited from our module, the lightweight network could receive an improvement of up to 32.6% in our experiment while maintaining its portability in the inference phase. The entire structure has been verified on two widely accepted public CT datasets LiTS17 and KiTS19. We demonstrate that a lightweight network distilled by our method has non-negligible value in the scenario which requires relatively high operating speed and low storage usage.

Related papers

• MDFI-Net: Multiscale Differential Feature Interaction Network for Accurate Retinal Vessel Segmentation [3.152646316470194]
  This paper proposes a feature-enhanced interaction network based on DPCN, named MDFI-Net. The proposed MDFI-Net achieves segmentation performance superior to state-of-the-art methods on public datasets.
  arXiv  Detail & Related papers  (2024-10-20T16:42:22Z)
• Slicer Networks [8.43960865813102]
  We propose the Slicer Network, a novel architecture for medical image analysis. The Slicer Network strategically refines and upsamples feature maps via a splatting-blurring-slicing process. Experiments across different medical imaging applications have verified the Slicer Network's improved accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-01-18T09:50:26Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• CMUNeXt: An Efficient Medical Image Segmentation Network based on Large Kernel and Skip Fusion [11.434576556863934]
  CMUNeXt is an efficient fully convolutional lightweight medical image segmentation network. It enables fast and accurate auxiliary diagnosis in real scene scenarios.
  arXiv  Detail & Related papers  (2023-08-02T15:54:00Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• UNesT: Local Spatial Representation Learning with Hierarchical Transformer for Efficient Medical Segmentation [29.287521185541298]
  We show that UNesT consistently achieves state-of-the-art performance and evaluate its generalizability and data efficiency. We show that UNesT consistently achieves state-of-the-art performance and evaluate its generalizability and data efficiency.
  arXiv  Detail & Related papers  (2022-09-28T19:14:38Z)
• InDuDoNet+: A Model-Driven Interpretable Dual Domain Network for Metal Artifact Reduction in CT Images [53.4351366246531]
  We construct a novel interpretable dual domain network, termed InDuDoNet+, into which CT imaging process is finely embedded. We analyze the CT values among different tissues, and merge the prior observations into a prior network for our InDuDoNet+, which significantly improve its generalization performance.
  arXiv  Detail & Related papers  (2021-12-23T15:52:37Z)
• HistoTransfer: Understanding Transfer Learning for Histopathology [9.231495418218813]
  We compare the performance of features extracted from networks trained on ImageNet and histopathology data. We investigate if features learned using more complex networks lead to gain in performance.
  arXiv  Detail & Related papers  (2021-06-13T18:55:23Z)
• Medical Transformer: Gated Axial-Attention for Medical Image Segmentation [73.98974074534497]
  We study the feasibility of using Transformer-based network architectures for medical image segmentation tasks. We propose a Gated Axial-Attention model which extends the existing architectures by introducing an additional control mechanism in the self-attention module. To train the model effectively on medical images, we propose a Local-Global training strategy (LoGo) which further improves the performance.
  arXiv  Detail & Related papers  (2021-02-21T18:35:14Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.