Statistical Modeling of Soft Error Influence on Neural Networks
- URL: http://arxiv.org/abs/2210.05876v1
- Date: Wed, 12 Oct 2022 02:28:21 GMT
- Title: Statistical Modeling of Soft Error Influence on Neural Networks
- Authors: Haitong Huang, Xinghua Xue, Cheng Liu, Ying Wang, Tao Luo, Long Cheng,
Huawei Li, Xiaowei Li
- Abstract summary: We develop a series of statistical models to analyze the behavior of NN models under soft errors in general.
The statistical models reveal not only the correlation between soft errors and NN model accuracy, but also how NN parameters such as quantization and architecture affect the reliability of NNs.
- Score: 12.298356981085316
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Soft errors in large VLSI circuits pose dramatic influence on computing- and
memory-intensive neural network (NN) processing. Understanding the influence of
soft errors on NNs is critical to protect against soft errors for reliable NN
processing. Prior work mainly rely on fault simulation to analyze the influence
of soft errors on NN processing. They are accurate but usually specific to
limited configurations of errors and NN models due to the prohibitively slow
simulation speed especially for large NN models and datasets. With the
observation that the influence of soft errors propagates across a large number
of neurons and accumulates as well, we propose to characterize the soft error
induced data disturbance on each neuron with normal distribution model
according to central limit theorem and develop a series of statistical models
to analyze the behavior of NN models under soft errors in general. The
statistical models reveal not only the correlation between soft errors and NN
model accuracy, but also how NN parameters such as quantization and
architecture affect the reliability of NNs. The proposed models are compared
with fault simulation and verified comprehensively. In addition, we observe
that the statistical models that characterize the soft error influence can also
be utilized to predict fault simulation results in many cases and we explore
the use of the proposed statistical models to accelerate fault simulations of
NNs. According to our experiments, the accelerated fault simulation shows
almost two orders of magnitude speedup with negligible simulation accuracy loss
over the baseline fault simulations.
Related papers
- Physics-informed neural networks (PINNs) for numerical model error approximation and superresolution [3.4393226199074114]
We propose physics-informed neural networks (PINNs) for simultaneous numerical model error approximation and superresolution.
PINNs effectively predict model errors in both x and y displacement fields with small differences between predictions and ground truth.
Our findings demonstrate that the integration of physics-informed loss functions enables neural networks (NNs) to surpass a purely data-driven approach for approximating model errors.
arXiv Detail & Related papers (2024-11-14T17:03:09Z) - Towards Long-Term predictions of Turbulence using Neural Operators [68.8204255655161]
It aims to develop reduced-order/surrogate models for turbulent flow simulations using Machine Learning.
Different model structures are analyzed, with U-NET structures performing better than the standard FNO in accuracy and stability.
arXiv Detail & Related papers (2023-07-25T14:09:53Z) - A critical look at deep neural network for dynamic system modeling [0.0]
This paper questions the capability of (deep) neural networks for the modeling of dynamic systems using input-output data.
For the identification of linear time-invariant (LTI) dynamic systems, two representative neural network models are compared.
For the LTI system, both LSTM and CFNN fail to deliver consistent models even in noise-free cases.
arXiv Detail & Related papers (2023-01-27T09:03:05Z) - Fast and Accurate Error Simulation for CNNs against Soft Errors [64.54260986994163]
We present a framework for the reliability analysis of Conal Neural Networks (CNNs) via an error simulation engine.
These error models are defined based on the corruption patterns of the output of the CNN operators induced by faults.
We show that our methodology achieves about 99% accuracy of the fault effects w.r.t. SASSIFI, and a speedup ranging from 44x up to 63x w.r.t.FI, that only implements a limited set of error models.
arXiv Detail & Related papers (2022-06-04T19:45:02Z) - Probabilistic model-error assessment of deep learning proxies: an
application to real-time inversion of borehole electromagnetic measurements [0.0]
We study the effects of the approximate nature of the deep learned models and associated model errors during the inversion of extra-deep borehole electromagnetic (EM) measurements.
Using a deep neural network (DNN) as a forward model allows us to perform thousands of model evaluations within seconds.
We present numerical results highlighting the challenges associated with the inversion of EM measurements while neglecting model error.
arXiv Detail & Related papers (2022-05-25T11:44:48Z) - Generalization of Neural Combinatorial Solvers Through the Lens of
Adversarial Robustness [68.97830259849086]
Most datasets only capture a simpler subproblem and likely suffer from spurious features.
We study adversarial robustness - a local generalization property - to reveal hard, model-specific instances and spurious features.
Unlike in other applications, where perturbation models are designed around subjective notions of imperceptibility, our perturbation models are efficient and sound.
Surprisingly, with such perturbations, a sufficiently expressive neural solver does not suffer from the limitations of the accuracy-robustness trade-off common in supervised learning.
arXiv Detail & Related papers (2021-10-21T07:28:11Z) - Characterizing possible failure modes in physics-informed neural
networks [55.83255669840384]
Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models.
We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs.
We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
arXiv Detail & Related papers (2021-09-02T16:06:45Z) - On the benefits of robust models in modulation recognition [53.391095789289736]
Deep Neural Networks (DNNs) using convolutional layers are state-of-the-art in many tasks in communications.
In other domains, like image classification, DNNs have been shown to be vulnerable to adversarial perturbations.
We propose a novel framework to test the robustness of current state-of-the-art models.
arXiv Detail & Related papers (2021-03-27T19:58:06Z) - Statistical model-based evaluation of neural networks [74.10854783437351]
We develop an experimental setup for the evaluation of neural networks (NNs)
The setup helps to benchmark a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds.
This allows us to test the effects of training data size, data dimension, data geometry, noise, and mismatch between training and testing conditions.
arXiv Detail & Related papers (2020-11-18T00:33:24Z) - Entropy-Based Modeling for Estimating Soft Errors Impact on Binarized
Neural Network Inference [2.249916681499244]
We present the relatively-accurate statistical models to delineate the impact of both undertaken single-event upset (SEU) and multi-bit upset (MBU) across layers and per each layer of the selected convolution neural network.
These models can be used for evaluating the error-resiliency magnitude of NN topology before adopting them in the safety-critical applications.
arXiv Detail & Related papers (2020-04-10T16:10:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.