Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators

• URL: http://arxiv.org/abs/2409.18553v1
• Date: Fri, 27 Sep 2024 08:45:55 GMT
• Title: Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators
• Authors: Seyedarmin Azizi, Mohammad Erfan Sadeghi, Mehdi Kamal, Massoud Pedram,
• Abstract summary: We model process-induced and aging-related variations of analog computing components on the accuracy of the analog neural networks. We introduce a denoising block inserted between selected layers of a pre-trained model. We demonstrate that training the denoising block significantly increases the model's robustness against various noise levels.
• Score: 4.416800723562206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose a framework to enhance the robustness of the neural models by mitigating the effects of process-induced and aging-related variations of analog computing components on the accuracy of the analog neural networks. We model these variations as the noise affecting the precision of the activations and introduce a denoising block inserted between selected layers of a pre-trained model. We demonstrate that training the denoising block significantly increases the model's robustness against various noise levels. To minimize the overhead associated with adding these blocks, we present an exploration algorithm to identify optimal insertion points for the denoising blocks. Additionally, we propose a specialized architecture to efficiently execute the denoising blocks, which can be integrated into mixed-signal accelerators. We evaluate the effectiveness of our approach using Deep Neural Network (DNN) models trained on the ImageNet and CIFAR-10 datasets. The results show that on average, by accepting 2.03% parameter count overhead, the accuracy drop due to the variations reduces from 31.7% to 1.15%.

Related papers

• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Walking Noise: On Layer-Specific Robustness of Neural Architectures against Noisy Computations and Associated Characteristic Learning Dynamics [1.5184189132709105]
  We discuss the implications of additive, multiplicative and mixed noise for different classification tasks and model architectures. We propose a methodology called Walking Noise which injects layer-specific noise to measure the robustness. We conclude with a discussion of the use of this methodology in practice, among others, discussing its use for tailored multi-execution in noisy environments.
  arXiv  Detail & Related papers  (2022-12-20T17:09:08Z)
• Multi-stage image denoising with the wavelet transform [125.2251438120701]
  Deep convolutional neural networks (CNNs) are used for image denoising via automatically mining accurate structure information. We propose a multi-stage image denoising CNN with the wavelet transform (MWDCNN) via three stages, i.e., a dynamic convolutional block (DCB), two cascaded wavelet transform and enhancement blocks (WEBs) and residual block (RB)
  arXiv  Detail & Related papers  (2022-09-26T03:28:23Z)
• Guided Diffusion Model for Adversarial Purification [103.4596751105955]
  Adversarial attacks disturb deep neural networks (DNNs) in various algorithms and frameworks. We propose a novel purification approach, referred to as guided diffusion model for purification (GDMP) On our comprehensive experiments across various datasets, the proposed GDMP is shown to reduce the perturbations raised by adversarial attacks to a shallow range.
  arXiv  Detail & Related papers  (2022-05-30T10:11:15Z)
• MANet: Improving Video Denoising with a Multi-Alignment Network [72.93429911044903]
  We present a multi-alignment network, which generates multiple flow proposals followed by attention-based averaging. Experiments on a large-scale video dataset demonstrate that our method improves the denoising baseline model by 0.2dB.
  arXiv  Detail & Related papers  (2022-02-20T00:52:07Z)
• Removing Noise from Extracellular Neural Recordings Using Fully Convolutional Denoising Autoencoders [62.997667081978825]
  We propose a Fully Convolutional Denoising Autoencoder, which learns to produce a clean neuronal activity signal from a noisy multichannel input. The experimental results on simulated data show that our proposed method can improve significantly the quality of noise-corrupted neural signals.
  arXiv  Detail & Related papers  (2021-09-18T14:51:24Z)
• Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections [73.95786440318369]
  We focus on the so-called implicit effect' of GNIs, which is the effect of the injected noise on the dynamics of gradient descent (SGD) We show that this effect induces an asymmetric heavy-tailed noise on gradient updates. We then formally prove that GNIs induce an implicit bias', which varies depending on the heaviness of the tails and the level of asymmetry.
  arXiv  Detail & Related papers  (2021-02-13T21:28:09Z)
• Image Denoising using Attention-Residual Convolutional Neural Networks [0.0]
  We propose a new learning-based non-blind denoising technique named Attention Residual Convolutional Neural Network (ARCNN) and its extension to blind denoising named Flexible Attention Residual Convolutional Neural Network (FARCNN) ARCNN achieved an overall average PSNR results of around 0.44dB and 0.96dB for Gaussian and Poisson denoising, respectively FARCNN presented very consistent results, even with slightly worsen performance compared to ARCNN.
  arXiv  Detail & Related papers  (2021-01-19T16:37:57Z)
• Improved Detection of Adversarial Images Using Deep Neural Networks [2.3993545400014873]
  Recent studies indicate that machine learning models used for classification tasks are vulnerable to adversarial examples. We propose a new approach called Feature Map Denoising to detect the adversarial inputs. We show the performance of detection on a mixed dataset consisting of adversarial examples.
  arXiv  Detail & Related papers  (2020-07-10T19:02:24Z)
• Robust Processing-In-Memory Neural Networks via Noise-Aware Normalization [26.270754571140735]
  PIM accelerators often suffer from intrinsic noise in the physical components. We propose a noise-agnostic method to achieve robust neural network performance against any noise setting.
  arXiv  Detail & Related papers  (2020-07-07T06:51:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.