Early-Exit with Class Exclusion for Efficient Inference of Neural Networks

• URL: http://arxiv.org/abs/2309.13443v2
• Date: Sat, 17 Feb 2024 08:03:00 GMT
• Title: Early-Exit with Class Exclusion for Efficient Inference of Neural Networks
• Authors: Jingcun Wang, Bing Li, Grace Li Zhang
• Abstract summary: We propose a class-based early-exit for dynamic inference in deep neural networks (DNNs) We take advantage of the learned features in these layers to exclude as many irrelevant classes as possible. Experimental results demonstrate the computational cost of DNNs in inference can be reduced significantly.
• Score: 4.180653524441411
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep neural networks (DNNs) have been successfully applied in various fields. In DNNs, a large number of multiply-accumulate (MAC) operations are required to be performed, posing critical challenges in applying them in resource-constrained platforms, e.g., edge devices. To address this challenge, in this paper, we propose a class-based early-exit for dynamic inference. Instead of pushing DNNs to make a dynamic decision at intermediate layers, we take advantage of the learned features in these layers to exclude as many irrelevant classes as possible, so that later layers only have to determine the target class among the remaining classes. When only one class remains at a layer, this class is the corresponding classification result. Experimental results demonstrate the computational cost of DNNs in inference can be reduced significantly with the proposed early-exit technique. The codes can be found at https://github.com/HWAI-TUDa/EarlyClassExclusion.

Related papers

• Class-Aware Pruning for Efficient Neural Networks [5.918784236241883]
  Pruning has been introduced to reduce the computational cost in executing deep neural networks (DNNs) In this paper, we propose a class-aware pruning technique to compress DNNs. Experimental results confirm that this class-aware pruning technique can significantly reduce the number of weights and FLOPs.
  arXiv  Detail & Related papers  (2023-12-10T13:07:54Z)
• SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference [20.564198591600647]
  Deep neural networks (DNNs) demonstrate outstanding performance across most computer vision tasks. Some critical applications, such as autonomous driving or medical imaging, also require investigation into their behavior. DNN attribution consists in studying the relationship between the predictions of a DNN and its inputs.
  arXiv  Detail & Related papers  (2023-08-09T07:45:51Z)
• Detecting Novelties with Empty Classes [6.953730499849023]
  We build upon anomaly detection to retrieve out-of-distribution (OoD) data as candidates for new classes. We introduce two loss functions, which 1) entice the DNN to assign OoD samples to the empty classes and 2) to minimize the inner-class feature distances between them.
  arXiv  Detail & Related papers  (2023-04-30T19:52:47Z)
• Rethinking Nearest Neighbors for Visual Classification [56.00783095670361]
  k-NN is a lazy learning method that aggregates the distance between the test image and top-k neighbors in a training set. We adopt k-NN with pre-trained visual representations produced by either supervised or self-supervised methods in two steps. Via extensive experiments on a wide range of classification tasks, our study reveals the generality and flexibility of k-NN integration.
  arXiv  Detail & Related papers  (2021-12-15T20:15:01Z)
• Understanding and Improving Early Stopping for Learning with Noisy Labels [63.0730063791198]
  The memorization effect of deep neural network (DNN) plays a pivotal role in many state-of-the-art label-noise learning methods. Current methods generally decide the early stopping point by considering a DNN as a whole. We propose to separate a DNN into different parts and progressively train them to address this problem.
  arXiv  Detail & Related papers  (2021-06-30T07:18:00Z)
• Online Limited Memory Neural-Linear Bandits with Likelihood Matching [53.18698496031658]
  We study neural-linear bandits for solving problems where both exploration and representation learning play an important role. We propose a likelihood matching algorithm that is resilient to catastrophic forgetting and is completely online.
  arXiv  Detail & Related papers  (2021-02-07T14:19:07Z)
• Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation [75.93960390191262]
  We exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes. We propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models.
  arXiv  Detail & Related papers  (2020-08-13T03:52:37Z)
• Many-Class Few-Shot Learning on Multi-Granularity Class Hierarchy [57.68486382473194]
  We study many-class few-shot (MCFS) problem in both supervised learning and meta-learning settings. In this paper, we leverage the class hierarchy as a prior knowledge to train a coarse-to-fine classifier. The model, "memory-augmented hierarchical-classification network (MahiNet)", performs coarse-to-fine classification where each coarse class can cover multiple fine classes.
  arXiv  Detail & Related papers  (2020-06-28T01:11:34Z)
• Robust Pruning at Initialization [61.30574156442608]
  A growing need for smaller, energy-efficient, neural networks to be able to use machine learning applications on devices with limited computational resources. For Deep NNs, such procedures remain unsatisfactory as the resulting pruned networks can be difficult to train and, for instance, they do not prevent one layer from being fully pruned.
  arXiv  Detail & Related papers  (2020-02-19T17:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.