PruningBench: A Comprehensive Benchmark of Structural Pruning

• URL: http://arxiv.org/abs/2406.12315v3
• Date: Sat, 20 Jul 2024 10:56:31 GMT
• Title: PruningBench: A Comprehensive Benchmark of Structural Pruning
• Authors: Haoling Li, Changhao Li, Mengqi Xue, Gongfan Fang, Sheng Zhou, Zunlei Feng, Huiqiong Wang, Yong Wang, Lechao Cheng, Mingli Song, Jie Song,
• Abstract summary: We present the first comprehensive benchmark, termed textitPruningBench, for structural pruning. PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques. It provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards.
• Score: 50.23493036025595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structural pruning has emerged as a promising approach for producing more efficient models. Nevertheless, the community suffers from a lack of standardized benchmarks and metrics, leaving the progress in this area not fully comprehended. To fill this gap, we present the first comprehensive benchmark, termed \textit{PruningBench}, for structural pruning. PruningBench showcases the following three characteristics: 1) PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques; 2) PruningBench systematically evaluates 16 existing pruning methods, encompassing a wide array of models (e.g., CNNs and ViTs) and tasks (e.g., classification and detection); 3) PruningBench provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards. We provide an online pruning platform http://pruning.vipazoo.cn for customizing pruning tasks and reproducing all results in this paper. Codes will be made publicly on https://github.com/HollyLee2000/PruningBench.

Related papers

• Protein Structure Tokenization: Benchmarking and New Recipe [16.842453216446987]
  We introduce StructTokenBench, a framework that comprehensively evaluates the quality and efficiency of structure tokenizers. We also develop AminoAseed, a strategy that enhances codebook updates and optimally balances codebook size and dimension for improved tokenizer utilization and quality.
  arXiv  Detail & Related papers  (2025-02-28T15:14:33Z)
• One-cycle Structured Pruning with Stability Driven Structure Search [20.18712941647407]
  Existing structured pruning typically involves multi-stage training procedures that often demand heavy computation. We propose an efficient framework for one-cycle structured pruning without compromising model performance. Our method achieves state-of-the-art accuracy while being one of the most efficient pruning frameworks in terms of training time.
  arXiv  Detail & Related papers  (2025-01-23T07:46:48Z)
• PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides [51.88536367177796]
  We propose a two-stage, edit-based approach inspired by human drafts for automatically generating presentations. PWTAgent first analyzes references to extract slide-level functional types and content schemas, then generates editing actions based on selected reference slides. PWTAgent significantly outperforms existing automatic presentation generation methods across all three dimensions.
  arXiv  Detail & Related papers  (2025-01-07T16:53:01Z)
• ShuffleBench: A Benchmark for Large-Scale Data Shuffling Operations with Distributed Stream Processing Frameworks [1.4374467687356276]
  This paper introduces ShuffleBench, a novel benchmark to evaluate the performance of modern stream processing frameworks. ShuffleBench is inspired by requirements for near real-time analytics of a large cloud observability platform. Our results show that Flink achieves the highest throughput while Hazelcast processes data streams with the lowest latency.
  arXiv  Detail & Related papers  (2024-03-07T15:06:24Z)
• RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems [43.797002322559834]
  RepoBench is a benchmark for evaluating code auto-completion systems. It consists of three evaluation tasks: RepoBench-R (Retrieval), RepoBench-C (Code Completion), and RepoBench-P (Pipeline)
  arXiv  Detail & Related papers  (2023-06-05T17:59:41Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Advancing Model Pruning via Bi-level Optimization [89.88761425199598]
  iterative magnitude pruning (IMP) is the predominant pruning method to successfully find 'winning tickets' One-shot pruning methods have been developed, but these schemes are usually unable to find winning tickets as good as IMP. We show that the proposed bi-level optimization-oriented pruning method (termed BiP) is a special class of BLO problems with a bi-linear problem structure.
  arXiv  Detail & Related papers  (2022-10-08T19:19:29Z)
• Benchopt: Reproducible, efficient and collaborative optimization benchmarks [67.29240500171532]
  Benchopt is a framework to automate, reproduce and publish optimization benchmarks in machine learning. Benchopt simplifies benchmarking for the community by providing an off-the-shelf tool for running, sharing and extending experiments.
  arXiv  Detail & Related papers  (2022-06-27T16:19:24Z)
• A Fast Post-Training Pruning Framework for Transformers [74.59556951906468]
  Pruning is an effective way to reduce the huge inference cost of large Transformer models. Prior work on model pruning requires retraining the model. We propose a fast post-training pruning framework for Transformers that does not require any retraining.
  arXiv  Detail & Related papers  (2022-03-29T07:41:11Z)
• Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking for Everyone [45.673814384050004]
  We introduce Codabench, an open-sourced, community-driven platform for benchmarking algorithms or software agents versus datasets or tasks. A public instance of Codabench is open to everyone, free of charge, and allows benchmark organizers to compare fairly submissions.
  arXiv  Detail & Related papers  (2021-10-12T07:54:34Z)
• MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models [78.45898846056303]
  Pruning is an effective method to reduce the memory footprint and computational cost associated with large natural language processing models. We develop a novel MultiLevel structured Pruning framework, which uses three different levels of structured pruning: head pruning, row pruning, and block-wise sparse pruning.
  arXiv  Detail & Related papers  (2021-05-30T22:00:44Z)
• Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot [55.37967301483917]
  Conventional wisdom of pruning algorithms suggests that pruning methods exploit information from training data to find goodworks. In this paper, we conduct sanity checks for the above beliefs on several recent unstructured pruning methods. We propose a series of simple emphdata-independent prune ratios for each layer, and randomly prune each layer accordingly to get a subnetwork.
  arXiv  Detail & Related papers  (2020-09-22T17:36:17Z)
• Image Matching across Wide Baselines: From Paper to Practice [80.9424750998559]
  We introduce a comprehensive benchmark for local features and robust estimation algorithms. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods. We show that with proper settings, classical solutions may still outperform the perceived state of the art.
  arXiv  Detail & Related papers  (2020-03-03T15:20:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.