Graph Flow: Cross-layer Graph Flow Distillation for Dual-Efficient Medical Image Segmentation

• URL: http://arxiv.org/abs/2203.08667v2
• Date: Thu, 17 Mar 2022 01:20:26 GMT
• Title: Graph Flow: Cross-layer Graph Flow Distillation for Dual-Efficient Medical Image Segmentation
• Authors: Wenxuan Zou, Muyi Sun
• Abstract summary: We propose Graph Flow, a novel comprehensive knowledge distillation method, to exploit the cross-layer graph flow knowledge for both network-efficient and annotation-efficient medical image segmentation. In this paper, we demonstrate the prominent ability of our method which state-of-the-art performance on different-modality and multi-category medical image datasets.
• Score: 0.76146285961466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the development of deep convolutional neural networks, medical image segmentation has achieved a series of breakthroughs in recent years. However, the higher-performance convolutional neural networks always mean numerous parameters and expensive computation costs, which will hinder the applications in clinical scenarios. Meanwhile, the scarceness of large-scale annotated medical image datasets further impedes the application of high-performance networks. To tackle these problems, we propose Graph Flow, a novel comprehensive knowledge distillation method, to exploit the cross-layer graph flow knowledge for both network-efficient and annotation-efficient medical image segmentation. Specifically, our Graph Flow Distillation constructs a variation graph which is employed to measure the flow of channel-wise salience features between different layers. Next, the knowledge included in the variation graph is transferred from a well-trained cumbersome teacher network to a non-trained compact student network. In addition, an unsupervised Paraphraser Module is designed to refine the knowledge of the teacher network, which is also beneficial for the stabilization of training procedure. Furthermore, we build a unified distillation framework by integrating the adversarial distillation and the vanilla logits distillation, which can further promote the final performance respectively. As a result, extensive experiments conducted on Gastric Cancer Segmentation Dataset and Synapse Multi-organ Segmentation Dataset demonstrate the prominent ability of our method which achieves state-of-the-art performance on these different-modality and multi-category medical image datasets. Moreover, we demonstrate the effectiveness of our Graph Flow through a new semi-supervised paradigm for dual-efficient medical image segmentation.

Related papers

• Predicting Stroke through Retinal Graphs and Multimodal Self-supervised Learning [0.46835339362676565]
  Early identification of stroke is crucial for intervention, requiring reliable models. We proposed an efficient retinal image representation together with clinical information to capture a comprehensive overview of cardiovascular health.
  arXiv  Detail & Related papers  (2024-11-08T14:40:56Z)
• Mew: Multiplexed Immunofluorescence Image Analysis through an Efficient Multiplex Network [84.88767228835928]
  We introduce Mew, a novel framework designed to efficiently process mIF images through the lens of multiplex network. Mew innovatively constructs a multiplex network comprising two distinct layers: a Voronoi network for geometric information and a Cell-type network for capturing cell-wise homogeneity. This framework equips a scalable and efficient Graph Neural Network (GNN), capable of processing the entire graph during training.
  arXiv  Detail & Related papers  (2024-07-25T08:22:30Z)
• Multi-Task Multi-Scale Contrastive Knowledge Distillation for Efficient Medical Image Segmentation [0.0]
  This thesis aims to investigate the feasibility of knowledge transfer between neural networks for medical image segmentation tasks. In the context of medical imaging, where the data volumes are often limited, leveraging knowledge from a larger pre-trained network could be useful.
  arXiv  Detail & Related papers  (2024-06-05T12:06:04Z)
• Graph Relation Distillation for Efficient Biomedical Instance Segmentation [80.51124447333493]
  We propose a graph relation distillation approach for efficient biomedical instance segmentation. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level. Experimental results on a number of biomedical datasets validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-01-12T04:41:23Z)
• Classification of developmental and brain disorders via graph convolutional aggregation [6.6356049194991815]
  We introduce an aggregator normalization graph convolutional network by leveraging aggregation in graph sampling. The proposed model learns discriminative graph node representations by incorporating both imaging and non-imaging features into the graph nodes and edges. We benchmark our model against several recent baseline methods on two large datasets, Autism Brain Imaging Data Exchange (ABIDE) and Alzheimer's Disease Neuroimaging Initiative (ADNI)
  arXiv  Detail & Related papers  (2023-11-13T14:36:29Z)
• Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model [64.29487107585665]
  Graph representation learning techniques on brain functional networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. Here, we propose an interpretable hierarchical signed graph representation learning model to extract graph-level representations from brain functional networks. In order to further improve the model performance, we also propose a new strategy to augment functional brain network data for contrastive learning.
  arXiv  Detail & Related papers  (2022-07-14T20:03:52Z)
• Efficient Medical Image Segmentation Based on Knowledge Distillation [30.857487609003197]
  We propose an efficient architecture by distilling knowledge from well-trained medical image segmentation networks to train another lightweight network. We also devise a novel distillation module tailored for medical image segmentation to transfer semantic region information from teacher to student network. We demonstrate that a lightweight network distilled by our method has non-negligible value in the scenario which requires relatively high operating speed and low storage usage.
  arXiv  Detail & Related papers  (2021-08-23T07:41:10Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)
• Graph-Based Neural Network Models with Multiple Self-Supervised Auxiliary Tasks [79.28094304325116]
  Graph Convolutional Networks are among the most promising approaches for capturing relationships among structured data points. We propose three novel self-supervised auxiliary tasks to train graph-based neural network models in a multi-task fashion.
  arXiv  Detail & Related papers  (2020-11-14T11:09:51Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.