A Quantization-Aware Training Based Lightweight Method for Neural Distinguishers

• URL: http://arxiv.org/abs/2603.05791v1
• Date: Fri, 06 Mar 2026 00:52:49 GMT
• Title: A Quantization-Aware Training Based Lightweight Method for Neural Distinguishers
• Abstract summary: In 2019, Gohr pioneered the application of deep neural networks to differential cryptanalysis.<n>This study proposes a lightweight neural distinguisher based on quantization-aware training.
• Score: 4.751864584128287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In 2019, Gohr pioneered the application of deep neural networks to differential cryptanalysis, developing DNN-based neural distinguisher classifiers to analyze the SPECK lightweight block cipher. Unlike traditional differential analysis, which relies on Boolean operations on 0-1 sequences, neural distinguishers extract continuous features, introducing 32-bit multiplications operations that increase complexity and potential redundancy. This study proposes a lightweight neural distinguisher based on quantization-aware training. Leveraging learnable step-size quantization, the model's weights are quantized to 1.58 bits, enabling the replacement of all convolutional multiplication operations with Boolean logic. Additionally, the ReLU activation function is reimplemented as a comparison-based indicator function. This transforms the original 32-bit multiplication-dependent architecture into a lightweight structure composed solely of Boolean operations, additions, and indicator functions. Experimental results confirm significant computational complexity reduction. Owing to a high proportion of zero-valued weights, the total operations amount to just 13.9% of Gohr's model. Critically, the most costly 32-bit multiplications are eliminated, with classification accuracy dropping by only 2.87%. When applied exclusively to the initial convolutional layer, the 128 1-by-1 convolutions are replaced with 4 Boolean operations on 16-bit sequences, incurring a negligible 0.3% accuracy loss.

Related papers

• SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits [6.860988566886594]
  We present textbfSparseKAN, a unified approach that compresses KANs along three complementary axes: basis functions, neurons/channels, and numerical precision.<n>SparseKAN equips the base branch, nonlinear basis branch, and individual basis terms with hierarchical learnable gates trained under a differentiable active-cost objective.
  arXiv  Detail & Related papers  (2026-08-01T20:42:41Z)
• Physics-Informed Learning of Effective Error Processes from Limited Noisy Transmon Measurements for Robust QAOA Reliability [0.0]
  We present a physics-informed pipeline for learning effective quantum error processes from finite-shot measurements.<n>The learned representations are compact effective models: local affine Bloch channels for each qubit and, in the three-qubit extension, pairwise residuals that capture correlated errors.<n>Results support effective error-process learning as a hardware-aware route toward more reliable variational quantum algorithms.
  arXiv  Detail & Related papers  (2026-05-29T20:58:00Z)
• KANtize: Exploring Low-bit Quantization of Kolmogorov-Arnold Networks for Efficient Inference [1.5102260054654923]
  Kolmogorov-Arnold Networks (KANs) have gained attention for their potential to outperform Multi-Layer Perceptrons (MLPs)<n>This study investigates the impact of low-bit quantization on KANs and its impact on computational complexity and hardware efficiency.
  arXiv  Detail & Related papers  (2026-03-18T00:32:11Z)
• Differentiable Logic Synthesis: Spectral Coefficient Selection via Sinkhorn-Constrained Composition [0.0]
  We introduce Hierarchical Spectral Composition, a differentiable architecture that selects spectral coefficients from a frozen Fourier basis.<n>We adapt this framework to logic synthesis, adding column-sign modulation to enable Boolean negation.
  arXiv  Detail & Related papers  (2026-01-20T13:26:52Z)
• Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics [64.62231094774211]
  Statefuls (e.g., Adam) maintain auxiliary information even 2x the model size in order to achieve optimal convergence.<n>SOLO enables Adam-styles to maintain quantized states with precision as low as 3 bits, or even 2 bits.<n>SOLO can thus be seamlessly applied to Adam-styles, leading to substantial memory savings with minimal accuracy loss.
  arXiv  Detail & Related papers  (2025-05-01T06:47:45Z)
• ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization [73.60493264901359]
  We present a unified framework for rigorous comparisons across 1-bit, 1.58-bit, 2-bit, 3-bit, and 4-bit quantization settings.<n>We show that ternary, 2-bit, and 3-bit quantization maintains comparable performance in the size-accuracy trade-off.<n>Considering hardware constraints, 2-bit quantization offers promising potential for memory reduction and speedup.
  arXiv  Detail & Related papers  (2025-02-04T18:59:26Z)
• Training Multi-Layer Binary Neural Networks With Local Binary Error Signals [3.7740044597960316]
  We propose a fully binary and gradient-free training algorithm for multi-layer Binary Neural Networks (BNNs)<n>The proposed algorithm relies on local binary error signals and binary weight updates, employing integer-valued hidden weights that serve as a synaptic metaplasticity mechanism.<n> Experimental results show test accuracy improvements of up to +35.47% over the only existing fully binary single-layer state-of-the-art solution.
  arXiv  Detail & Related papers  (2024-11-28T09:12:04Z)
• Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance [68.8204255655161]
  We introduce a quantization-aware training algorithm that guarantees avoiding numerical overflow when reducing the precision of accumulators during inference. We evaluate our algorithm across multiple quantized models that we train for different tasks, showing that our approach can reduce the precision of accumulators while maintaining model accuracy with respect to a floating-point baseline.
  arXiv  Detail & Related papers  (2023-01-31T02:46:57Z)
• Improved Differential-neural Cryptanalysis for Round-reduced Simeck32/64 [7.503547605892289]
  We develop the Inception neural network that is compatible with the round function of Simeck to improve the accuracy of neural distinguishers. Compared with the DDT-based distinguishers, the 9-, and 10-round neural distinguishers achieve better accuracy. An enhanced 15-round and the first practical 16-, and 17-round attacks are implemented for Simeck32/64, and the success rate of both the 15-, and 16-round attacks is almost 100%.
  arXiv  Detail & Related papers  (2023-01-27T09:00:09Z)
• 8-bit Optimizers via Block-wise Quantization [57.25800395197516]
  Statefuls maintain statistics over time, e.g., the exponentially smoothed sum (SGD with momentum) or squared sum (Adam) of past values. This state can be used to accelerate optimization compared to plain gradient descent but uses memory that might otherwise be allocated to model parameters. In this paper, we develop first gradients that use 8-bit statistics while maintaining the performance levels of using 32-bit gradient states.
  arXiv  Detail & Related papers  (2021-10-06T15:43:20Z)
• Searching for Low-Bit Weights in Quantized Neural Networks [129.8319019563356]
  Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. We present to regard the discrete weights in an arbitrary quantized neural network as searchable variables, and utilize a differential method to search them accurately.
  arXiv  Detail & Related papers  (2020-09-18T09:13:26Z)
• WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic [57.07483440807549]
  We propose a method that adapts neural networks to use low-resolution (8-bit) additions in the accumulators, achieving classification accuracy comparable to their 32-bit counterparts. We demonstrate the efficacy of our approach on both software and hardware platforms.
  arXiv  Detail & Related papers  (2020-07-26T23:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.