Approximate Multiplier Induced Error Propagation in Deep Neural Networks

• URL: http://arxiv.org/abs/2512.06537v1
• Date: Sat, 06 Dec 2025 19:05:17 GMT
• Title: Approximate Multiplier Induced Error Propagation in Deep Neural Networks
• Authors: A. M. H. H. Alahakoon, Hassaan Saadat, Darshana Jayasinghe, Sri Parameswaran,
• Abstract summary: This work presents an analytical framework that connects the statistical error moments of an AxM to the induced distortion in General Matrix multiplication.<n>By providing a lightweight alternative to behavioral or hardware level simulations, this framework enables rapid estimation of AxM impact on Deep Neural Networks (DNNs) inference quality.
• Score: 0.9099663022952496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Neural Networks (DNNs) rely heavily on dense arithmetic operations, motivating the use of Approximate Multipliers (AxMs) to reduce energy consumption in hardware accelerators. However, a rigorous mathematical characterization of how AxMs error distributions influence DNN accuracy remains underdeveloped. This work presents an analytical framework that connects the statistical error moments of an AxM to the induced distortion in General Matrix Multiplication (GEMM). Using the Frobenius norm of the resulting error matrix, we derive a closed form expression for practical DNN dimensions that demonstrates the distortion is predominantly governed by the multiplier mean error (bias). To evaluate this model in realistic settings, we incorporate controlled error injection into GEMM and convolution layers and examine its effect on ImageNet scale networks. The predicted distortion correlates strongly with the observed accuracy degradation, and an error configurable AxM case study implemented on an FPGA further confirms the analytical trends. By providing a lightweight alternative to behavioral or hardware level simulations, this framework enables rapid estimation of AxM impact on DNN inference quality.

Related papers

• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• An End-to-End Differentiable, Graph Neural Network-Embedded Pore Network Model for Permeability Prediction [0.42970700836450487]
  Pore network models (PNMs) estimate pore-scale hydraulic conductance, limiting their accuracy in complex structures.<n>We present an end-to-end differentiable hybrid framework that embeds a graph neural network (GNN) into a PNM.<n>The model achieves high accuracy and adjoins well across different scales, outperforming both pure data-driven and traditional PNM approaches.
  arXiv  Detail & Related papers  (2025-09-17T09:15:23Z)
• Dimension reduction for derivative-informed operator learning: An analysis of approximation errors [3.7051887945349518]
  We study the derivative-informed learning of nonlinear operators between infinite-dimensional separable Hilbert spaces by neural networks.<n>We analyze the approximation errors of neural operators in Sobolev norms over infinite-dimensional Gaussian input measures.
  arXiv  Detail & Related papers  (2025-04-11T17:56:52Z)
• Enhancing Accuracy in Deep Learning Using Random Matrix Theory [4.00671924018776]
  We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs) Our numerical results show that this pruning leads to a drastic reduction of parameters while not reducing the accuracy of DNNs and CNNs. Our results offer valuable insights into the practical application of RMT for the creation of more efficient and accurate deep-learning models.
  arXiv  Detail & Related papers  (2023-10-04T21:17:31Z)
• On Error Propagation of Diffusion Models [77.91480554418048]
  We develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs. We apply the cumulative error as a regularization term to reduce error propagation. Our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.
  arXiv  Detail & Related papers  (2023-08-09T15:31:17Z)
• Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks [63.614519238823206]
  Spiking Neural Networks (SNNs) are a practical choice when the target hardware reaches the edge of computing. We employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error.
  arXiv  Detail & Related papers  (2022-11-22T15:08:05Z)
• A Robust Backpropagation-Free Framework for Images [47.97322346441165]
  We present an error kernel driven activation alignment algorithm for image data. EKDAA accomplishes through the introduction of locally derived error transmission kernels and error maps. Results are presented for an EKDAA trained CNN that employs a non-differentiable activation function.
  arXiv  Detail & Related papers  (2022-06-03T21:14:10Z)
• MemSE: Fast MSE Prediction for Noisy Memristor-Based DNN Accelerators [5.553959304125023]
  We theoretically analyze the mean squared error of DNNs that use memristors to compute matrix-vector multiplications (MVM) We take into account both the quantization noise, due to the necessity of reducing the DNN model size, and the programming noise, stemming from the variability during the programming of the memristance value. The proposed method is almost two order of magnitude faster than Monte-Carlo simulation, thus making it possible to optimize the implementation parameters to achieve minimal error for a given power constraint.
  arXiv  Detail & Related papers  (2022-05-03T18:10:43Z)
• Shaping Deep Feature Space towards Gaussian Mixture for Visual Classification [74.48695037007306]
  We propose a Gaussian mixture (GM) loss function for deep neural networks for visual classification. With a classification margin and a likelihood regularization, the GM loss facilitates both high classification performance and accurate modeling of the feature distribution. The proposed model can be implemented easily and efficiently without using extra trainable parameters.
  arXiv  Detail & Related papers  (2020-11-18T03:32:27Z)
• On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression [79.86233860519621]
  We exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. We show that MAE can be interpreted as an error modeled by Laplacian distribution.
  arXiv  Detail & Related papers  (2020-08-12T22:41:26Z)
• Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression [79.86233860519621]
  We show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of error (MAE) is between a mean absolute error and an expected feature error. We propose a proposed upper bounds of MAE for DNN based vector-to-vector regression.
  arXiv  Detail & Related papers  (2020-08-04T19:39:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.