Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters

• URL: http://arxiv.org/abs/2009.12021v1
• Date: Fri, 25 Sep 2020 03:58:40 GMT
• Title: Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters
• Authors: Xudong Wang and Stella X. Yu
• Abstract summary: Convolution is the main building block of convolutional neural networks (CNN) We propose Tied Block Convolution (TBC) that shares the same thinner filters over equal blocks of channels and produces multiple responses with a single filter.
• Score: 50.10906063068743
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolution is the main building block of convolutional neural networks (CNN). We observe that an optimized CNN often has highly correlated filters as the number of channels increases with depth, reducing the expressive power of feature representations. We propose Tied Block Convolution (TBC) that shares the same thinner filters over equal blocks of channels and produces multiple responses with a single filter. The concept of TBC can also be extended to group convolution and fully connected layers, and can be applied to various backbone networks and attention modules. Our extensive experimentation on classification, detection, instance segmentation, and attention demonstrates TBC's significant across-the-board gain over standard convolution and group convolution. The proposed TiedSE attention module can even use 64 times fewer parameters than the SE module to achieve comparable performance. In particular, standard CNNs often fail to accurately aggregate information in the presence of occlusion and result in multiple redundant partial object proposals. By sharing filters across channels, TBC reduces correlation and can effectively handle highly overlapping instances. TBC increases the average precision for object detection on MS-COCO by 6% when the occlusion ratio is 80%. Our code will be released.

Related papers

• A Passive Similarity based CNN Filter Pruning for Efficient Acoustic Scene Classification [23.661189257759535]
  We present a method to develop low-complexity convolutional neural networks (CNNs) for acoustic scene classification (ASC) We propose a passive filter pruning framework, where a few convolutional filters from the CNNs are eliminated to yield compressed CNNs. The proposed method is simple, reduces computations per inference by 27%, with 25% fewer parameters, with less than 1% drop in accuracy.
  arXiv  Detail & Related papers  (2022-03-29T17:00:06Z)
• Decoupled Dynamic Filter Networks [85.38058820176047]
  We propose the Decoupled Dynamic Filter (DDF) that can simultaneously tackle both of these shortcomings. Inspired by recent advances in attention, DDF decouples a depth-wise dynamic filter into spatial and channel dynamic filters. We observe a significant boost in performance when replacing standard convolution with DDF in classification networks.
  arXiv  Detail & Related papers  (2021-04-29T04:55:33Z)
• Self-grouping Convolutional Neural Networks [30.732298624941738]
  We propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN. For each filter, we first evaluate the importance value of their input channels to identify the importance vectors. Using the resulting emphdata-dependent centroids, we prune the less important connections, which implicitly minimizes the accuracy loss of the pruning.
  arXiv  Detail & Related papers  (2020-09-29T06:24:32Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)
• Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters [64.46270549587004]
  Convolutional neural networks (CNNs) have been successfully used in a range of tasks. CNNs are often viewed as "black-box" and lack of interpretability. We propose a novel strategy to train interpretable CNNs by encouraging class-specific filters.
  arXiv  Detail & Related papers  (2020-07-16T09:12:26Z)
• PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer [76.44375136492827]
  Convolutional Neural Networks (CNNs) are often scale-sensitive. We bridge this regret by exploiting multi-scale features in a finer granularity. The proposed convolution operation, named Poly-Scale Convolution (PSConv), mixes up a spectrum of dilation rates.
  arXiv  Detail & Related papers  (2020-07-13T05:14:11Z)
• Multigrid-in-Channels Architectures for Wide Convolutional Neural Networks [6.929025509877642]
  We present a multigrid approach that combats the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs) Our examples from supervised image classification show that applying this strategy to residual networks and MobileNetV2 considerably reduces the number of parameters without negatively affecting accuracy.
  arXiv  Detail & Related papers  (2020-06-11T20:28:36Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.