Hardware Efficient Approximate Convolution with Tunable Error Tolerance for CNNs

• URL: http://arxiv.org/abs/2603.10100v1
• Date: Tue, 10 Mar 2026 17:59:10 GMT
• Title: Hardware Efficient Approximate Convolution with Tunable Error Tolerance for CNNs
• Abstract summary: We propose a soft sparsity'' paradigm using a hardware efficient Most Significant Bit proxy to skip negligible non-zero multiplications.<n>This method reduces ReLU MACs by 88.42% and Tanh MACs by 74.87% with zero accuracy loss--outperforming zero-skipping by 5x.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern CNNs' high computational demands hinder edge deployment, as traditional ``hard'' sparsity (skipping mathematical zeros) loses effectiveness in deep layers or with smooth activations like Tanh. We propose a ``soft sparsity'' paradigm using a hardware efficient Most Significant Bit (MSB) proxy to skip negligible non-zero multiplications. Integrated as a custom RISC-V instruction and evaluated on LeNet-5 (MNIST), this method reduces ReLU MACs by 88.42% and Tanh MACs by 74.87% with zero accuracy loss--outperforming zero-skipping by 5x. By clock-gating inactive multipliers, we estimate power savings of 35.2\% for ReLU and 29.96\% for Tanh. While memory access makes power reduction sub-linear to operation savings, this approach significantly optimizes resource-constrained inference.

Related papers

• Training-free sparse attention based on cumulative energy filtering [5.794925677954963]
  Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest.<n>We formulate the token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation.<n>Our algorithm is deeply integrated with Flash Attention (FA), eliminating the need for any additional masking overhead.
  arXiv  Detail & Related papers  (2026-06-15T07:26:57Z)
• KANtize: Exploring Low-bit Quantization of Kolmogorov-Arnold Networks for Efficient Inference [1.5102260054654923]
  Kolmogorov-Arnold Networks (KANs) have gained attention for their potential to outperform Multi-Layer Perceptrons (MLPs)<n>This study investigates the impact of low-bit quantization on KANs and its impact on computational complexity and hardware efficiency.
  arXiv  Detail & Related papers  (2026-03-18T00:32:11Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• BLaST: High Performance Inference and Pretraining using BLock Sparse Transformers [18.53326467276254]
  We introduce (Bl)ock (a)nd (S)parse (T)ransformers (BLaST))<n>BLaST achieves up to 95% sparsity in matrix multiplication with negligible accuracy loss.<n>We show a 2.2x inference speed Llama 3.2 with 16 GPUs with 4.45x reduction in inference memory footprint.
  arXiv  Detail & Related papers  (2025-07-03T18:53:54Z)
• Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis [9.884521812433661]
  Quaff is a Quantized parameter-efficient fine-tuning framework for large language models.<n>It suppresses outliers exclusively invariant channels using lightweight operations.<n>It achieves a 1.73x latency reduction and 30% memory savings over full-precision fine-tuning.
  arXiv  Detail & Related papers  (2025-05-20T07:19:36Z)
• Scalable LLM Math Reasoning Acceleration with Low-rank Distillation [57.922185576872444]
  We propose a resource-efficient distillation method to recover lost capabilities from deploying efficient inference methods.<n>With original weights unperturbed, roughly 1% of additional parameters, and only 20K synthetic training samples, we are able to recover much if not all of the math capabilities lost.
  arXiv  Detail & Related papers  (2025-05-08T17:51:24Z)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers [52.199303258423306]
  We propose a novel density loss that encourages higher activation sparsity in pre-trained models. Our proposed method, textbfDEFT, can consistently reduce activation density by up to textbf44.94% on RoBERTa$_mathrmLarge$ and by textbf53.19% (encoder density) and textbf90.60% (decoder density) on Flan-T5$_mathrmXXL$.
  arXiv  Detail & Related papers  (2024-02-02T21:25:46Z)
• Hadamard Domain Training with Integers for Class Incremental Quantized Learning [1.4416751609100908]
  Continual learning can be cost-prohibitive for resource-constraint edge platforms. We propose a technique that transforms to enable low-precision training with only integer matrix multiplications. We achieve less than 0.5% and 3% accuracy degradation while we quantize all matrix multiplications inputs down to 4-bits with 8-bit accumulators.
  arXiv  Detail & Related papers  (2023-10-05T16:52:59Z)
• Data-Free Dynamic Compression of CNNs for Tractable Efficiency [46.498278084317704]
  structured pruning approaches have shown promise in lowering floating-point operations without substantial drops in accuracy.<n>We propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning.<n>We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy.
  arXiv  Detail & Related papers  (2023-09-29T13:09:40Z)
• Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance [68.8204255655161]
  We introduce a quantization-aware training algorithm that guarantees avoiding numerical overflow when reducing the precision of accumulators during inference. We evaluate our algorithm across multiple quantized models that we train for different tasks, showing that our approach can reduce the precision of accumulators while maintaining model accuracy with respect to a floating-point baseline.
  arXiv  Detail & Related papers  (2023-01-31T02:46:57Z)
• MicroNet: Improving Image Recognition with Extremely Low FLOPs [82.54764264255505]
  We find two factors, sparse connectivity and dynamic activation function, are effective to improve the accuracy. We present a new dynamic activation function, named Dynamic Shift Max, to improve the non-linearity. We arrive at a family of networks, named MicroNet, that achieves significant performance gains over the state of the art in the low FLOP regime.
  arXiv  Detail & Related papers  (2021-08-12T17:59:41Z)
• Non-Parametric Adaptive Network Pruning [125.4414216272874]
  We introduce non-parametric modeling to simplify the algorithm design. Inspired by the face recognition community, we use a message passing algorithm to obtain an adaptive number of exemplars. EPruner breaks the dependency on the training data in determining the "important" filters.
  arXiv  Detail & Related papers  (2021-01-20T06:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.