Expanding-and-Shrinking Binary Neural Networks

• URL: http://arxiv.org/abs/2503.23709v1
• Date: Mon, 31 Mar 2025 04:13:33 GMT
• Title: Expanding-and-Shrinking Binary Neural Networks
• Authors: Xulong Shi, Caiyi Sun, Zhi Qi, Liu Hao, Xiaodong Yang,
• Abstract summary: We propose the expanding-and-shrinking operation, which enhances binary feature maps with negligible increase of complexity.<n>Our approach generalizes well across diverse applications ranging from image classification, object detection to generative diffusion model.
• Score: 3.6593289731343246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While binary neural networks (BNNs) offer significant benefits in terms of speed, memory and energy, they encounter substantial accuracy degradation in challenging tasks compared to their real-valued counterparts. Due to the binarization of weights and activations, the possible values of each entry in the feature maps generated by BNNs are strongly constrained. To tackle this limitation, we propose the expanding-and-shrinking operation, which enhances binary feature maps with negligible increase of computation complexity, thereby strengthening the representation capacity. Extensive experiments conducted on multiple benchmarks reveal that our approach generalizes well across diverse applications ranging from image classification, object detection to generative diffusion model, while also achieving remarkable improvement over various leading binarization algorithms based on different architectures including both CNNs and Transformers.

Related papers

• BHViT: Binarized Hybrid Vision Transformer [53.38894971164072]
  Model binarization has made significant progress in enabling real-time and energy-efficient computation for convolutional neural networks (CNN)<n>We propose BHViT, a binarization-friendly hybrid ViT architecture and its full binarization model with the guidance of three important observations.<n>Our proposed algorithm achieves SOTA performance among binary ViT methods.
  arXiv  Detail & Related papers  (2025-03-04T08:35:01Z)
• BiDense: Binarization for Dense Prediction [62.70804353158387]
  BiDense is a generalized binary neural network (BNN) designed for efficient and accurate dense prediction tasks. BiDense incorporates two key techniques: the Distribution-adaptive Binarizer (DAB) and the Channel-adaptive Full-precision Bypass (CFB)
  arXiv  Detail & Related papers  (2024-11-15T16:46:04Z)
• SO(2) and O(2) Equivariance in Image Recognition with Bessel-Convolutional Neural Networks [63.24965775030674]
  This work presents the development of Bessel-convolutional neural networks (B-CNNs) B-CNNs exploit a particular decomposition based on Bessel functions to modify the key operation between images and filters. Study is carried out to assess the performances of B-CNNs compared to other methods.
  arXiv  Detail & Related papers  (2023-04-18T18:06:35Z)
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness [15.29595828816055]
  This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. We use Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm. Experiments show that our proposal achieves promising results in all the objectives, and direct relation are presented.
  arXiv  Detail & Related papers  (2023-02-20T19:33:38Z)
• Network Binarization via Contrastive Learning [16.274341164897827]
  We establish a novel contrastive learning framework while training Binary Neural Networks (BNNs) MI is introduced as the metric to measure the information shared between binary and FP activations. Results show that our method can be implemented as a pile-up module on existing state-of-the-art binarization methods.
  arXiv  Detail & Related papers  (2022-07-06T21:04:53Z)
• Distribution-sensitive Information Retention for Accurate Binary Neural Network [49.971345958676196]
  We present a novel Distribution-sensitive Information Retention Network (DIR-Net) to retain the information of the forward activations and backward gradients. Our DIR-Net consistently outperforms the SOTA binarization approaches under mainstream and compact architectures. We conduct our DIR-Net on real-world resource-limited devices which achieves 11.1 times storage saving and 5.4 times speedup.
  arXiv  Detail & Related papers  (2021-09-25T10:59:39Z)
• BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning [37.376989855065545]
  Spiking Neural Networks (SNNs) are biologically inspired, dynamic, event-driven models that enhance energy efficiency. An SNN model is introduced that combines the benefits of temporally sparse binary activations and of binary weights. Experiments validate the performance loss with respect to full-precision implementations.
  arXiv  Detail & Related papers  (2020-12-15T14:06:36Z)
• High-Capacity Expert Binary Networks [56.87581500474093]
  Network binarization is a promising hardware-aware direction for creating efficient deep models. Despite its memory and computational advantages, reducing the accuracy gap between binary models and their real-valued counterparts remains an unsolved challenging research problem. We propose Expert Binary Convolution, which, for the first time, tailors conditional computing to binary networks by learning to select one data-specific expert binary filter at a time conditioned on input features.
  arXiv  Detail & Related papers  (2020-10-07T17:58:10Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)
• Exploring the Connection Between Binary and Spiking Neural Networks [1.329054857829016]
  We bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets.
  arXiv  Detail & Related papers  (2020-02-24T03:46:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.