A Low-cost Fault Corrector for Deep Neural Networks through Range Restriction

• URL: http://arxiv.org/abs/2003.13874v4
• Date: Mon, 29 Mar 2021 01:47:53 GMT
• Title: A Low-cost Fault Corrector for Deep Neural Networks through Range Restriction
• Authors: Zitao Chen, Guanpeng Li and Karthik Pattabiraman
• Abstract summary: Deep neural networks (DNNs) in safety-critical domains have engendered serious reliability concerns. This work proposes Ranger, a low-cost fault corrector, which directly rectifies the faulty output due to transient faults without re-computation.
• Score: 1.8907108368038215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The adoption of deep neural networks (DNNs) in safety-critical domains has engendered serious reliability concerns. A prominent example is hardware transient faults that are growing in frequency due to the progressive technology scaling, and can lead to failures in DNNs. This work proposes Ranger, a low-cost fault corrector, which directly rectifies the faulty output due to transient faults without re-computation. DNNs are inherently resilient to benign faults (which will not cause output corruption), but not to critical faults (which can result in erroneous output). Ranger is an automated transformation to selectively restrict the value ranges in DNNs, which reduces the large deviations caused by critical faults and transforms them to benign faults that can be tolerated by the inherent resilience of the DNNs. Our evaluation on 8 DNNs demonstrates Ranger significantly increases the error resilience of the DNNs (by 3x to 50x), with no loss in accuracy, and with negligible overheads.

Related papers

• Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning [0.4660328753262075]
  This paper introduces a model-level hardening approach for CNNs by integrating error correction directly into the neural networks. The proposed method demonstrates fault resilience nearly equivalent to TMR-based correction but with significantly reduced overhead. Remarkably, the hardened pruned CNNs perform up to 24% faster than the hardened un-pruned ones.
  arXiv  Detail & Related papers  (2024-05-17T09:42:44Z)
• ELEGANT: Certified Defense on the Fairness of Graph Neural Networks [94.10433608311604]
  Graph Neural Networks (GNNs) have emerged as a prominent graph learning model in various graph-based tasks. malicious attackers could easily corrupt the fairness level of their predictions by adding perturbations to the input graph data. We propose a principled framework named ELEGANT to study a novel problem of certifiable defense on the fairness level of GNNs.
  arXiv  Detail & Related papers  (2023-11-05T20:29:40Z)
• CRAFT: Criticality-Aware Fault-Tolerance Enhancement Techniques for Emerging Memories-Based Deep Neural Networks [7.566423455230909]
  Deep Neural Networks (DNNs) have emerged as the most effective programming paradigm for computer vision and natural language processing applications. This paper proposes CRAFT, i.e., Criticality-Aware Fault-Tolerance Enhancement Techniques to enhance the reliability of NVM-based DNNs.
  arXiv  Detail & Related papers  (2023-02-08T03:39:11Z)
• The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks [94.63547069706459]
  #DNN-Verification problem involves counting the number of input configurations of a DNN that result in a violation of a safety property. We propose a novel approach that returns the exact count of violations. We present experimental results on a set of safety-critical benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T18:32:01Z)
• Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators [6.8082132475259405]
  This paper focuses on quantifying the accuracy given that a transient error has occurred, which tells us how well a network behaves when a transient error occurs. We show that existing Resiliency Accuracy (RA) formulation is fundamentally inaccurate, because it incorrectly assumes that software variables have equal faulty probability under hardware transient faults. We present an algorithm that captures the faulty probabilities of DNN variables under transient faults and, thus, provides correct RA estimations validated by hardware.
  arXiv  Detail & Related papers  (2022-12-05T23:16:20Z)
• Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness [172.61581010141978]
  Certifiable robustness is a desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios. We propose a novel solution to strategically manipulate neurons, by "grafting" appropriate levels of linearity.
  arXiv  Detail & Related papers  (2022-06-15T22:42:29Z)
• Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead [67.87678914831477]
  Deep Neural Networks (DNNs) enable a wide series of technological advancements. Recent findings indicate that transient hardware faults may corrupt the models prediction dramatically. In this work, we propose to tackle the reliability issue both at training and model design time.
  arXiv  Detail & Related papers  (2022-05-28T13:09:30Z)
• FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions [0.05249805590164901]
  Deep neural networks (DNNs) are increasingly being deployed in safety-critical systems such as personal healthcare devices and self-driving cars. In this paper, we propose FitAct, a low-cost approach to enhance the error resilience of DNNs by deploying fine-grained post-trainable activation functions.
  arXiv  Detail & Related papers  (2021-12-27T07:07:50Z)
• A Biased Graph Neural Network Sampler with Near-Optimal Regret [57.70126763759996]
  Graph neural networks (GNN) have emerged as a vehicle for applying deep network architectures to graph and relational data. In this paper, we build upon existing work and treat GNN neighbor sampling as a multi-armed bandit problem. We introduce a newly-designed reward function that introduces some degree of bias designed to reduce variance and avoid unstable, possibly-unbounded payouts.
  arXiv  Detail & Related papers  (2021-03-01T15:55:58Z)
• Exploring Fault-Energy Trade-offs in Approximate DNN Hardware Accelerators [2.9649783577150837]
  We present an extensive layer-wise and bit-wise fault resilience and energy analysis of different AxDNNs. Our results demonstrate that the fault resilience in AxDNNs is to the energy efficiency.
  arXiv  Detail & Related papers  (2021-01-08T05:52:12Z)
• GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examples [77.99182201815763]
  Deep neural networks (DNNs) are vulnerable to adversarial examples and other data perturbations. GraN is a time- and parameter-efficient method that is easily adaptable to any DNN. GraN achieves state-of-the-art performance on numerous problem set-ups.
  arXiv  Detail & Related papers  (2020-04-20T10:09:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.