Self-Distribution Binary Neural Networks
- URL: http://arxiv.org/abs/2103.02394v2
- Date: Thu, 4 Mar 2021 02:17:03 GMT
- Title: Self-Distribution Binary Neural Networks
- Authors: Ping Xue, Yang Lu, Jingfei Chang, Xing Wei, Zhen Wei
- Abstract summary: We study the binary neural networks (BNNs) of which both the weights and activations are binary (i.e., 1-bit representation)
We propose Self-Distribution Binary Neural Network (SD-BNN)
Experiments on CIFAR-10 and ImageNet datasets show that the proposed SD-BNN consistently outperforms the state-of-the-art (SOTA) BNNs.
- Score: 18.69165083747967
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we study the binary neural networks (BNNs) of which both the
weights and activations are binary (i.e., 1-bit representation). Feature
representation is critical for deep neural networks, while in BNNs, the
features only differ in signs. Prior work introduces scaling factors into
binary weights and activations to reduce the quantization error and effectively
improves the classification accuracy of BNNs. However, the scaling factors not
only increase the computational complexity of networks, but also make no sense
to the signs of binary features. To this end, Self-Distribution Binary Neural
Network (SD-BNN) is proposed. Firstly, we utilize Activation Self Distribution
(ASD) to adaptively adjust the sign distribution of activations, thereby
improve the sign differences of the outputs of the convolution. Secondly, we
adjust the sign distribution of weights through Weight Self Distribution (WSD)
and then fine-tune the sign distribution of the outputs of the convolution.
Extensive experiments on CIFAR-10 and ImageNet datasets with various network
structures show that the proposed SD-BNN consistently outperforms the
state-of-the-art (SOTA) BNNs (e.g., achieves 92.5% on CIFAR-10 and 66.5% on
ImageNet with ResNet-18) with less computation cost. Code is available at
https://github.com/ pingxue-hfut/SD-BNN.
Related papers
Err
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.