Meta Pruning via Graph Metanetworks : A Meta Learning Framework for Network Pruning

• URL: http://arxiv.org/abs/2506.12041v1
• Date: Sat, 24 May 2025 08:22:34 GMT
• Title: Meta Pruning via Graph Metanetworks : A Meta Learning Framework for Network Pruning
• Authors: Yewei Liu, Xiyuan Wang, Muhan Zhang,
• Abstract summary: We propose a novel approach in which we "use a neural network to prune neural networks"<n>A metanetwork is a network that takes another network as input and produces a modified network as output.<n>We train a metanetwork that learns the pruning strategy automatically which can transform a network that is hard to prune into another network that is much easier to prune.
• Score: 24.505897569096476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Network pruning, aimed at reducing network size while preserving accuracy, has attracted significant research interest. Numerous pruning techniques have been proposed over time. They are becoming increasingly effective, but more complex and harder to interpret as well. Given the inherent complexity of neural networks, we argue that manually designing pruning criteria has reached a bottleneck. To address this, we propose a novel approach in which we "use a neural network to prune neural networks". More specifically, we introduce the newly developed idea of metanetwork from meta-learning into pruning. A metanetwork is a network that takes another network as input and produces a modified network as output. In this paper, we first establish a bijective mapping between neural networks and graphs, and then employ a graph neural network as our metanetwork. We train a metanetwork that learns the pruning strategy automatically which can transform a network that is hard to prune into another network that is much easier to prune. Once the metanetwork is trained, our pruning needs nothing more than a feedforward through the metanetwork and the standard finetuning to prune at state-of-the-art. Our method achieved outstanding results on many popular and representative pruning tasks (including ResNet56 on CIFAR10, VGG19 on CIFAR100, ResNet50 on ImageNet). Our code is available at https://github.com/Yewei-Liu/MetaPruning

Related papers

• Stitching for Neuroevolution: Recombining Deep Neural Networks without Breaking Them [0.0]
  Traditional approaches to neuroevolution often start from scratch. Recombining trained networks is non-trivial because architectures and feature representations typically differ. We employ stitching, which merges the networks by introducing new layers at crossover points.
  arXiv  Detail & Related papers  (2024-03-21T08:30:44Z)
• You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets [105.24703398193843]
  Untrainedworks in graph neural networks (GNNs) still remains mysterious. We show that the found untrainedworks can substantially mitigate the GNN over-smoothing problem. We also observe that such sparse untrainedworks have appealing performance in out-of-distribution detection and robustness of input perturbations.
  arXiv  Detail & Related papers  (2022-11-28T14:17:36Z)
• Multipath Graph Convolutional Neural Networks [6.216778442751621]
  We propose a novel Multipath Graph convolutional neural network that aggregates the output of multiple different shallow networks. Results show that the proposed method not only attains increased accuracy but also requires fewer training epochs to converge.
  arXiv  Detail & Related papers  (2021-05-04T14:11:20Z)
• Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks [50.684661759340145]
  Firefly neural architecture descent is a general framework for progressively and dynamically growing neural networks. We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource-efficient neural architectures. In particular, it learns networks that are smaller in size but have higher average accuracy than those learned by the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-17T04:47:18Z)
• Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks [78.65792427542672]
  Dynamic Graph Network (DG-Net) is a complete directed acyclic graph, where the nodes represent convolutional blocks and the edges represent connection paths. Instead of using the same path of the network, DG-Net aggregates features dynamically in each node, which allows the network to have more representation ability.
  arXiv  Detail & Related papers  (2020-10-02T16:50:26Z)
• Graph Prototypical Networks for Few-shot Learning on Attributed Networks [72.31180045017835]
  We propose a graph meta-learning framework -- Graph Prototypical Networks (GPN) GPN is able to perform textitmeta-learning on an attributed network and derive a highly generalizable model for handling the target classification task.
  arXiv  Detail & Related papers  (2020-06-23T04:13:23Z)
• A Framework for Neural Network Pruning Using Gibbs Distributions [34.0576955010317]
  Gibbs pruning is a novel framework for expressing and designing neural network pruning methods. It can train and prune a network simultaneously in such a way that the learned weights and pruning mask are well-adapted for each other. We achieve a new state-of-the-art result for pruning ResNet-56 with the CIFAR-10 dataset.
  arXiv  Detail & Related papers  (2020-06-08T23:04:53Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)
• Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection [35.121856435677564]
  We propose a simple greedy selection approach for finding goodworks in deep neural networks. Applying the greedy selection strategy on sufficiently large pre-trained networks guarantees to find smallworks with lower loss than networks directly trained with gradient descent.
  arXiv  Detail & Related papers  (2020-03-03T21:03:11Z)
• Knapsack Pruning with Inner Distillation [11.04321604965426]
  We propose a novel pruning method that optimize the final accuracy of the pruned network. We prune the network channels while maintaining the high-level structure of the network. Our method leads to state-of-the-art pruning results on ImageNet, CIFAR-10 and CIFAR-100 using ResNet backbones.
  arXiv  Detail & Related papers  (2020-02-19T16:04:48Z)
• A "Network Pruning Network" Approach to Deep Model Compression [62.68120664998911]
  We present a filter pruning approach for deep model compression using a multitask network. Our approach is based on learning a a pruner network to prune a pre-trained target network. The compressed model produced by our approach is generic and does not need any special hardware/software support.
  arXiv  Detail & Related papers  (2020-01-15T20:38:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.