XSepConv: Extremely Separated Convolution

• URL: http://arxiv.org/abs/2002.12046v1
• Date: Thu, 27 Feb 2020 11:46:17 GMT
• Title: XSepConv: Extremely Separated Convolution
• Authors: Jiarong Chen, Zongqing Lu, Jing-Hao Xue, Qingmin Liao
• Abstract summary: We propose a novel extremely separated convolutional block (XSepConv) It fuses spatially separable convolutions into depthwise convolution to reduce both the computational cost and parameter size of large kernels. XSepConv is designed to be an efficient alternative to vanilla depthwise convolution with large kernel sizes.
• Score: 60.90871656244126
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Depthwise convolution has gradually become an indispensable operation for modern efficient neural networks and larger kernel sizes ($\ge5$) have been applied to it recently. In this paper, we propose a novel extremely separated convolutional block (XSepConv), which fuses spatially separable convolutions into depthwise convolution to further reduce both the computational cost and parameter size of large kernels. Furthermore, an extra $2\times2$ depthwise convolution coupled with improved symmetric padding strategy is employed to compensate for the side effect brought by spatially separable convolutions. XSepConv is designed to be an efficient alternative to vanilla depthwise convolution with large kernel sizes. To verify this, we use XSepConv for the state-of-the-art architecture MobileNetV3-Small and carry out extensive experiments on four highly competitive benchmark datasets (CIFAR-10, CIFAR-100, SVHN and Tiny-ImageNet) to demonstrate that XSepConv can indeed strike a better trade-off between accuracy and efficiency.

Related papers

• InceptionNeXt: When Inception Meets ConvNeXt [167.61042926444105]
  We build a series of networks, namely IncepitonNeXt, which not only enjoy high throughputs but also maintain competitive performance. InceptionNeXt achieves 1.6x higher training throughputs than ConvNeX-T, as well as attains 0.2% top-1 accuracy improvement on ImageNet-1K.
  arXiv  Detail & Related papers  (2023-03-29T17:59:58Z)
• GMConv: Modulating Effective Receptive Fields for Convolutional Kernels [52.50351140755224]
  In convolutional neural networks, the convolutions are performed using a square kernel with a fixed N $times$ N receptive field (RF) Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv) in this work. Our GMConv can directly replace the standard convolutions in existing CNNs and can be easily trained end-to-end by standard back-propagation.
  arXiv  Detail & Related papers  (2023-02-09T10:17:17Z)
• Learnable Heterogeneous Convolution: Learning both topology and strength [0.5801044612920815]
  Learnable Heterogeneous Convolution realizes joint learning of kernel shape and weights. A model based on our method can converge with structural sparse weights. Our method either reduces VGG16/19 and ResNet34/50 computation by nearly 5x on CIFAR10 and 2x on ImageNet without harming the performance.
  arXiv  Detail & Related papers  (2023-01-13T08:48:12Z)
• Omni-Dimensional Dynamic Convolution [25.78940854339179]
  Learning a single static convolutional kernel in each convolutional layer is the common training paradigm of modern Convolutional Neural Networks (CNNs) Recent research in dynamic convolution shows that learning a linear combination of $n$ convolutional kernels weighted with their input-dependent attentions can significantly improve the accuracy of light-weight CNNs. We present Omni-dimensional Dynamic Convolution (ODConv), a more generalized yet elegant dynamic convolution design.
  arXiv  Detail & Related papers  (2022-09-16T14:05:38Z)
• FuSeConv: Fully Separable Convolutions for Fast Inference on Systolic Arrays [2.8583189395674653]
  We propose FuSeConv as a drop-in replacement for depth-wise separable convolution. FuSeConv generalizes the decomposition of convolutions fully to separable 1D convolutions along spatial and depth dimensions. We achieve a significant speed-up of 3x-7x with the MobileNet family of networks on a systolic array of size 64x64, with comparable accuracy on the ImageNet dataset.
  arXiv  Detail & Related papers  (2021-05-27T20:19:39Z)
• Hardware Architecture of Embedded Inference Accelerator and Analysis of Algorithms for Depthwise and Large-Kernel Convolutions [27.141754658998323]
  The proposed architecture can support filter kernels with different sizes with high flexibility. For image classification, the accuracy is increased by 1% by simply replacing $3 times 3$ filters with $5 times 5$ filters in depthwise convolutions.
  arXiv  Detail & Related papers  (2021-04-29T05:45:16Z)
• PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer [76.44375136492827]
  Convolutional Neural Networks (CNNs) are often scale-sensitive. We bridge this regret by exploiting multi-scale features in a finer granularity. The proposed convolution operation, named Poly-Scale Convolution (PSConv), mixes up a spectrum of dilation rates.
  arXiv  Detail & Related papers  (2020-07-13T05:14:11Z)
• DO-Conv: Depthwise Over-parameterized Convolutional Layer [66.46704754669169]
  We propose to augment a convolutional layer with an additional depthwise convolution, where each input channel is convolved with a different 2D kernel. We show with extensive experiments that the mere replacement of conventional convolutional layers with DO-Conv layers boosts the performance of CNNs.
  arXiv  Detail & Related papers  (2020-06-22T06:57:10Z)
• Dynamic Region-Aware Convolution [85.20099799084026]
  We propose a new convolution called Dynamic Region-Aware Convolution (DRConv), which can automatically assign multiple filters to corresponding spatial regions. On ImageNet classification, DRConv-based ShuffleNetV2-0.5x achieves state-of-the-art performance of 67.1% at 46M multiply-adds level with 6.3% relative improvement.
  arXiv  Detail & Related papers  (2020-03-27T05:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.