From Bit-Position Sensitivity to Unequal Error Protection for DNN Inference Memory

• URL: http://arxiv.org/abs/2607.19623v1
• Date: Tue, 21 Jul 2026 23:07:56 GMT
• Title: From Bit-Position Sensitivity to Unequal Error Protection for DNN Inference Memory
• Abstract summary: We characterize per-bit-position fault sensitivity in ML inference across 16 workloads.<n>Our central empirical finding is a sharp bit-sensitivity transition: flipping any of the least-significant fraction bits up to a data-type-specific threshold, Xsafe, degrades task metrics by less than 1% under deterministic single-bit stress tests.
• Score: 0.4610618579893946
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We characterize per-bit-position fault sensitivity in ML inference across 16 workloads -- spanning transformer-based models and attention-free CNNs -- and across three floating-point formats. Our central empirical finding is a sharp bit-sensitivity transition: flipping any of the least-significant fraction bits up to a data-type-specific threshold, Xsafe, degrades task metrics by less than 1% under deterministic single-bit stress tests. Sensitivity rises through the upper fraction bits and spikes at the exponent-mantissa boundary, where a single-bit flip causes catastrophic collapse. Because low-order bits are largely inconsequential while high-order and exponent bits are critical, uniform SECDED protection -- which guards every bit equally at 12.5% storage overhead -- is unnecessarily conservative. We derive per-data-type Xsafe floors (FP16: 6, BF16: 4, FP32: 15) and workload-aware tiers that widen the unprotected region for resilient model classes, raising ECC savings to 37.5-62.5% without retraining. Text-conditioned diffusion models dictate the conservative floor; vision encoders, NLU models, and resilient LLMs tolerate wider bypass regions. These floors and tiers drive an Unequal Error Protection (UEP) codec with per-cacheline data-type tags and a dual-partition SRAM architecture for ML accelerators. Validation across 870+ fault-injection runs confirms selective protection holds under contiguous 2- and 3-bit upsets. The codec reduces ECC area by 27.8% relative to uniform SECDED; dual-voltage operation of the non-critical partition lowers gross BF16 read energy by about 17%, with a roughly 4% dual-partition macro-area overhead.

Related papers

• PRISM: Sensitivity-Aware PolynoMial PRuning for EffIcient Neural Network Encryption [0.0]
  This paper presents a systematic reliability characterization of pruned CKKS-encritical neural networks.<n>Polynomial-Sensitivity-Aware Pruning (PSAP) is a structured pruning method that is inherently reliability-aware.
  arXiv  Detail & Related papers  (2026-07-20T05:47:06Z)
• Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL [18.134050827462005]
  Large Language Models (LLMs) deploy into mission-critical domains (e.g. finance, medicine, law)<n>Experiments with 16-bit precisions still exhibit catastrophic output divergence across heterogeneous GPUs.<n>We propose Hybrid Error ALleviation (HEAL), a targeted intervention that approximates FP32 accelerators while resolving hardware constraints.
  arXiv  Detail & Related papers  (2026-06-19T01:21:10Z)
• Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation [6.129872931808218]
  Key-value ( KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory.<n>In this study, we explore alignment preservation under KV cache quantization.<n>We find that low-bit quantization can silently destroy safety alignment.
  arXiv  Detail & Related papers  (2026-06-01T02:02:20Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• Runtime-Certified Bounded-Error Quantized Attention [0.0]
  We present a tiered KV cache architecture that enables runtime-certified attention.<n>A two-term error decomposition yields per-head, per-step bounds on (i) attention distribution distortion from key quantization and (ii) value reconstruction error.<n>Across PG-19, NIAH, and RULER benchmarks, the system matches dense FP16 KV quality within noise for language modelling and retrieval tasks.
  arXiv  Detail & Related papers  (2026-05-20T08:04:40Z)
• XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inference [0.0]
  XFP determines codebook size, outlier budget, and packing per layer automatically.<n>XFP reaches 138 tok/s single-stream decode on workstation hardware.<n>For models that do not fit in the target memory envelope, we present the H-Process.
  arXiv  Detail & Related papers  (2026-05-14T13:52:31Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels [83.99688944263843]
  Weight-De Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction.<n>At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working memory in bf16.<n>A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r2) intermediates, eliminating the dense product.
  arXiv  Detail & Related papers  (2026-03-23T17:57:24Z)
• Speeding Up MACE: Low-Precision Tricks for Equivarient Force Fields [51.95157731126864]
  Machine-learning force fields can deliver accurate molecular dynamics (MD) at high computational cost.<n>This thesis aims to make MACE cheaper and faster by identifying computational bottlenecks and evaluating low-precision execution policies.
  arXiv  Detail & Related papers  (2025-10-23T14:02:34Z)
• Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding [144.70522923640095]
  Large language models (LLMs) face low hardware efficiency during decoding.<n>This paper introduces Step-3, a hardware-aware model-system co-design optimized for minimizing decoding costs.<n>Step-3 significantly reduces theoretical decoding costs compared with models like DeepSeek-V3 and Qwen3 MoE 235B.
  arXiv  Detail & Related papers  (2025-07-25T16:53:13Z)
• Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask [74.64216073678617]
  AMD performs parallel NAR inference within contiguous blocks of output labels concealed using attention masks. A beam search algorithm is designed to leverage a dynamic fusion of CTC, AR Decoder, and AMD probabilities. Experiments on the LibriSpeech-100hr corpus suggest the tripartite Decoder incorporating the AMD module produces a maximum decoding speed-up ratio of 1.73x.
  arXiv  Detail & Related papers  (2024-06-14T13:42:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.