EfficientNet-eLite: Extremely Lightweight and Efficient CNN Models for Edge Devices by Network Candidate Search

• URL: http://arxiv.org/abs/2009.07409v1
• Date: Wed, 16 Sep 2020 01:11:10 GMT
• Title: EfficientNet-eLite: Extremely Lightweight and Efficient CNN Models for Edge Devices by Network Candidate Search
• Authors: Ching-Chen Wang, Ching-Te Chiu, Jheng-Yi Chang
• Abstract summary: We propose a novel of Network Candidate Search (NCS) to study the trade-off between the resource usage and the performance. In our experiment, we collect candidate CNN models from EfficientNet-B0 to be scaled down in varied way through width, depth, input resolution and compound scaling down. For further embracing the CNN edge application with Application-Specific Integrated Circuit (ASIC), we adjust the architectures of EfficientNet-eLite to build the more hardware-friendly version, EfficientNet-HF.
• Score: 13.467017642143583
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embedding Convolutional Neural Network (CNN) into edge devices for inference is a very challenging task because such lightweight hardware is not born to handle this heavyweight software, which is the common overhead from the modern state-of-the-art CNN models. In this paper, targeting at reducing the overhead with trading the accuracy as less as possible, we propose a novel of Network Candidate Search (NCS), an alternative way to study the trade-off between the resource usage and the performance through grouping concepts and elimination tournament. Besides, NCS can also be generalized across any neural network. In our experiment, we collect candidate CNN models from EfficientNet-B0 to be scaled down in varied way through width, depth, input resolution and compound scaling down, applying NCS to research the scaling-down trade-off. Meanwhile, a family of extremely lightweight EfficientNet is obtained, called EfficientNet-eLite. For further embracing the CNN edge application with Application-Specific Integrated Circuit (ASIC), we adjust the architectures of EfficientNet-eLite to build the more hardware-friendly version, EfficientNet-HF. Evaluation on ImageNet dataset, both proposed EfficientNet-eLite and EfficientNet-HF present better parameter usage and accuracy than the previous start-of-the-art CNNs. Particularly, the smallest member of EfficientNet-eLite is more lightweight than the best and smallest existing MnasNet with 1.46x less parameters and 0.56% higher accuracy. Code is available at https://github.com/Ching-Chen-Wang/EfficientNet-eLite

Related papers

• Lightweight Pixel Difference Networks for Efficient Visual Representation Learning [20.345417462198725]
  We propose two novel types of convolutions, dubbed emphPixel Difference Convolution (PDC) and Binary PDC (Bi-PDC) We present two lightweight deep networks named emphPixel Difference Networks (PiDiNet) and emph Binary PiDiNet (Bi-PiDiNet) respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition.
  arXiv  Detail & Related papers  (2024-02-01T08:39:31Z)
• Enhancing Small Object Encoding in Deep Neural Networks: Introducing Fast&Focused-Net with Volume-wise Dot Product Layer [0.0]
  We introduce Fast&Focused-Net, a novel deep neural network architecture tailored for encoding small objects into fixed-length feature vectors. Fast&Focused-Net employs a series of our newly proposed layer, the Volume-wise Dot Product (VDP) layer, designed to address several inherent limitations of CNNs. For small object classification tasks, our network outperformed state-of-the-art methods on datasets such as CIFAR-10, CIFAR-100, STL-10, SVHN-Cropped, and Fashion-MNIST. In the context of larger image classification, when combined with a transformer encoder (ViT
  arXiv  Detail & Related papers  (2024-01-18T09:31:25Z)
• MogaNet: Multi-order Gated Aggregation Network [64.16774341908365]
  We propose a new family of modern ConvNets, dubbed MogaNet, for discriminative visual representation learning. MogaNet encapsulates conceptually simple yet effective convolutions and gated aggregation into a compact module. MogaNet exhibits great scalability, impressive efficiency of parameters, and competitive performance compared to state-of-the-art ViTs and ConvNets on ImageNet.
  arXiv  Detail & Related papers  (2022-11-07T04:31:17Z)
• BCNet: Searching for Network Width with Bilaterally Coupled Network [56.14248440683152]
  We introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Our method achieves state-of-the-art or competing performance over other baseline methods.
  arXiv  Detail & Related papers  (2021-05-21T18:54:03Z)
• Lite-HRNet: A Lightweight High-Resolution Network [97.17242913089464]
  We present an efficient high-resolution network, Lite-HRNet, for human pose estimation. We find that heavily-used pointwise (1x1) convolutions in shuffle blocks become the computational bottleneck. We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks.
  arXiv  Detail & Related papers  (2021-04-13T17:59:31Z)
• AdderNet and its Minimalist Hardware Design for Energy-Efficient Artificial Intelligence [111.09105910265154]
  We present a novel minimalist hardware architecture using adder convolutional neural network (AdderNet) The whole AdderNet can practically achieve 16% enhancement in speed. We conclude the AdderNet is able to surpass all the other competitors.
  arXiv  Detail & Related papers  (2021-01-25T11:31:52Z)
• Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets [65.28292822614418]
  Giant formula for simultaneously enlarging the resolution, depth and width provides us a Rubik's cube for neural networks. This paper aims to explore the twisting rules for obtaining deep neural networks with minimum model sizes and computational costs.
  arXiv  Detail & Related papers  (2020-10-28T08:49:45Z)
• LRNNet: A Light-Weighted Network with Efficient Reduced Non-Local Operation for Real-Time Semantic Segmentation [15.010572800399057]
  This paper introduces a light-weighted network with an efficient reduced non-local module (LRNNet) for efficient and realtime semantic segmentation. Experiments demonstrate our superior trade-off among light-weight, speed, computation and accuracy.
  arXiv  Detail & Related papers  (2020-06-04T08:55:15Z)
• DRU-net: An Efficient Deep Convolutional Neural Network for Medical Image Segmentation [2.3574651879602215]
  Residual network (ResNet) and densely connected network (DenseNet) have significantly improved the training efficiency and performance of deep convolutional neural networks (DCNNs) We propose an efficient network architecture by considering advantages of both networks.
  arXiv  Detail & Related papers  (2020-04-28T12:16:24Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.