KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and Quantization

• URL: http://arxiv.org/abs/2011.14691v1
• Date: Mon, 30 Nov 2020 11:11:22 GMT
• Title: KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and Quantization
• Authors: Het Shah, Avishree Khare, Neelay Shah, Khizir Siddiqui
• Abstract summary: We present KD-Lib, an open-source PyTorch based library. It contains state-of-the-art modular implementations of algorithms from the three families on top of multiple abstraction layers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, the growing size of neural networks has led to a vast amount of research concerning compression techniques to mitigate the drawbacks of such large sizes. Most of these research works can be categorized into three broad families : Knowledge Distillation, Pruning, and Quantization. While there has been steady research in this domain, adoption and commercial usage of the proposed techniques has not quite progressed at the rate. We present KD-Lib, an open-source PyTorch based library, which contains state-of-the-art modular implementations of algorithms from the three families on top of multiple abstraction layers. KD-Lib is model and algorithm-agnostic, with extended support for hyperparameter tuning using Optuna and Tensorboard for logging and monitoring. The library can be found at - https://github.com/SforAiDl/KD_Lib.

Related papers

• pyvene: A Library for Understanding and Improving PyTorch Models via Interventions [79.72930339711478]
  $textbfpyvene$ is an open-source library that supports customizable interventions on a range of different PyTorch modules. We show how $textbfpyvene$ provides a unified framework for performing interventions on neural models and sharing the intervened upon models with others.
  arXiv  Detail & Related papers  (2024-03-12T16:46:54Z)
• LibAUC: A Deep Learning Library for X-Risk Optimization [43.32145407575245]
  This paper introduces the award-winning deep learning (DL) library called LibAUC. LibAUC implements state-of-the-art algorithms towards optimizing a family of risk functions named X-risks.
  arXiv  Detail & Related papers  (2023-06-05T17:43:46Z)
• SequeL: A Continual Learning Library in PyTorch and JAX [50.33956216274694]
  SequeL is a library for Continual Learning that supports both PyTorch and JAX frameworks. It provides a unified interface for a wide range of Continual Learning algorithms, including regularization-based approaches, replay-based approaches, and hybrid approaches. We release SequeL as an open-source library, enabling researchers and developers to easily experiment and extend the library for their own purposes.
  arXiv  Detail & Related papers  (2023-04-21T10:00:22Z)
• Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow [50.691232400959656]
  Trieste is an open-source Python package for Bayesian optimization and active learning. Our library enables the plug-and-play of popular models within sequential decision-making loops.
  arXiv  Detail & Related papers  (2023-02-16T17:21:49Z)
• TorchNTK: A Library for Calculation of Neural Tangent Kernels of PyTorch Models [16.30276204466139]
  We introduce torchNTK, a python library to calculate the empirical neural tangent kernel (NTK) of neural network models in the PyTorch framework. A feature of the library is that we expose the user to layerwise NTK components, and show that in some regimes a layerwise calculation is more memory efficient.
  arXiv  Detail & Related papers  (2022-05-24T21:27:58Z)
• PyGOD: A Python Library for Graph Outlier Detection [56.33769221859135]
  PyGOD is an open-source library for detecting outliers in graph data. It supports a wide array of leading graph-based methods for outlier detection. PyGOD is released under a BSD 2-Clause license at https://pygod.org and at the Python Package Index (PyPI)
  arXiv  Detail & Related papers  (2022-04-26T06:15:21Z)
• ADAPT : Awesome Domain Adaptation Python Toolbox [5.932280316886339]
  ADAPT is an open-source python library providing the implementation of several domain adaptation methods. The library is suited for scikit-learn estimator object (object which implement fit and predict methods) and tensorflow models.
  arXiv  Detail & Related papers  (2021-07-07T07:20:21Z)
• Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking [4.721069729610892]
  Deep reinforcement learning has been one of the fastest growing fields of machine learning over the past years and numerous libraries have been open sourced to support research. This paper introduces Tonic, a Python library allowing researchers to quickly implement new ideas and measure their importance.
  arXiv  Detail & Related papers  (2020-11-15T14:10:37Z)
• Captum: A unified and generic model interpretability library for PyTorch [49.72749684393332]
  We introduce a novel, unified, open-source model interpretability library for PyTorch. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms. It can be used for both classification and non-classification models.
  arXiv  Detail & Related papers  (2020-09-16T18:57:57Z)
• Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python [77.33905890197269]
  We describe a new library which implements a unified pathwise coordinate optimization for a variety of sparse learning problems. The library is coded in R++ and has user-friendly sparse experiments.
  arXiv  Detail & Related papers  (2020-06-27T02:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.