Designing DNNs for a trade-off between robustness and processing performance in embedded devices

• URL: http://arxiv.org/abs/2412.03682v1
• Date: Wed, 04 Dec 2024 19:34:33 GMT
• Title: Designing DNNs for a trade-off between robustness and processing performance in embedded devices
• Authors: Jon Gutiérrez-Zaballa, Koldo Basterretxea, Javier Echanobe,
• Abstract summary: Machine learning-based embedded systems need to be robust against soft errors.<n>This paper investigates the suitability of using bounded AFs to improve model robustness against perturbations.<n>We analyze encoder-decoder fully convolutional models aimed at performing semantic segmentation tasks on hyperspectral images for scene understanding in autonomous driving.
• Score: 1.474723404975345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning-based embedded systems employed in safety-critical applications such as aerospace and autonomous driving need to be robust against perturbations produced by soft errors. Soft errors are an increasing concern in modern digital processors since smaller transistor geometries and lower voltages give electronic devices a higher sensitivity to background radiation. The resilience of deep neural network (DNN) models to perturbations in their parameters is determined, to a large extent, by the structure of the model itself, and also by the selected numerical representation and used arithmetic precision. When compression techniques such as model pruning and model quantization are applied to reduce memory footprint and computational complexity for deployment, both model structure and numerical representation are modified and thus, soft error robustness also changes. In this sense, although the choice of activation functions (AFs) in DNN models is frequently ignored, it conditions not only their accuracy and trainability, but also compressibility rates and numerical robustness. This paper investigates the suitability of using bounded AFs to improve model robustness against DNN parameter perturbations, assessing at the same time the impact of this choice on deployment in terms of model accuracy, compressibility, and computational burden. In particular, we analyze encoder-decoder fully convolutional models aimed at performing semantic segmentation tasks on hyperspectral images for scene understanding in autonomous driving. Deployment characterization is performed experimentally on an AMD-Xilinx's KV260 SoM.

Related papers

• Balancing Robustness and Efficiency in Embedded DNNs Through Activation Function Selection [1.474723404975345]
  Machine learning-based embedded systems for safety-critical applications must be robust to perturbations caused by soft errors. We focus on encoder-decoder convolutional models developed for semantic segmentation of hyperspectral images with application to autonomous driving systems.
  arXiv  Detail & Related papers  (2025-04-07T14:21:31Z)
• Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems [49.819436680336786]
  We propose an efficient transformed Gaussian process state-space model (ETGPSSM) for scalable and flexible modeling of high-dimensional, non-stationary dynamical systems. Specifically, our ETGPSSM integrates a single shared GP with input-dependent normalizing flows, yielding an expressive implicit process prior that captures complex, non-stationary transition dynamics. Our ETGPSSM outperforms existing GPSSMs and neural network-based SSMs in terms of computational efficiency and accuracy.
  arXiv  Detail & Related papers  (2025-03-24T03:19:45Z)
• Reduced Order Modeling with Shallow Recurrent Decoder Networks [5.686433280542813]
  SHRED-ROM is a robust decoding-only strategy that encodes the numerically unstable approximation of an inverse. We show that SHRED-ROM accurately reconstructs the state dynamics for new parameter values starting from limited fixed or mobile sensors.
  arXiv  Detail & Related papers  (2025-02-15T23:41:31Z)
• Neural Network Modeling of Microstructure Complexity Using Digital Libraries [1.03590082373586]
  We evaluate the performance of artificial and spiking neural networks in learning and predicting fatigue crack growth and Turing pattern development. Our assessment suggests that the leaky integrate-and-fire neuron model offers superior predictive accuracy with fewer parameters and less memory usage.
  arXiv  Detail & Related papers  (2025-01-30T07:44:21Z)
• Evaluating Single Event Upsets in Deep Neural Networks for Semantic Segmentation: an embedded system perspective [1.474723404975345]
  This paper delves into the robustness assessment in embedded Deep Neural Networks (DNNs)<n>By scrutinizing the layer-by-layer and bit-by-bit sensitivity of various encoder-decoder models to soft errors, this study thoroughly investigates the vulnerability of segmentation DNNs to SEUs.<n>We propose a set of practical lightweight error mitigation techniques with no memory or computational cost suitable for resource-constrained deployments.
  arXiv  Detail & Related papers  (2024-12-04T18:28:38Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control [45.84205238554709]
  We present a method for reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC.
  arXiv  Detail & Related papers  (2023-08-03T10:21:53Z)
• Evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems [0.0]
  Uncertainty Quantification (UQ) is a self assessed estimate of the model error. We examine several machine learning techniques, including both Gaussian processes and a family UQ-augmented neural networks. We evaluate UQ accuracy (distinct from model accuracy) using two metrics: the distribution of normalized residuals on validation data, and the distribution of estimated uncertainties.
  arXiv  Detail & Related papers  (2023-06-27T02:35:25Z)
• RAMP-Net: A Robust Adaptive MPC for Quadrotors via Physics-informed Neural Network [6.309365332210523]
  We propose a Robust Adaptive MPC framework via PINNs (RAMP-Net), which uses a neural network trained partly from simple ODEs and partly from data. We report 7.8% to 43.2% and 8.04% to 61.5% reduction in tracking errors for speeds ranging from 0.5 to 1.75 m/s compared to two SOTA regression based MPC methods.
  arXiv  Detail & Related papers  (2022-09-19T16:11:51Z)
• Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs [70.51212431290611]
  Partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. We propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-04-13T08:53:41Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• A data-driven peridynamic continuum model for upscaling molecular dynamics [3.1196544696082613]
  We propose a learning framework to extract, from molecular dynamics data, an optimal Linear Peridynamic Solid model. We provide sufficient well-posedness conditions for discretized LPS models with sign-changing influence functions. This framework guarantees that the resulting model is mathematically well-posed, physically consistent, and that it generalizes well to settings that are different from the ones used during training.
  arXiv  Detail & Related papers  (2021-08-04T07:07:47Z)
• Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search [25.841113960607334]
  Emerging device-based Computing-in-memory (CiM) has been proved to be a promising candidate for high-energy efficiency deep neural network (DNN) computations. Most emerging devices suffer uncertainty issues, resulting in a difference between actual data stored and the weight value it is designed to be. This leads to an accuracy drop from trained models to actually deployed platforms.
  arXiv  Detail & Related papers  (2021-07-06T23:29:36Z)
• Rate Distortion Characteristic Modeling for Neural Image Compression [59.25700168404325]
  End-to-end optimization capability offers neural image compression (NIC) superior lossy compression performance. distinct models are required to be trained to reach different points in the rate-distortion (R-D) space. We make efforts to formulate the essential mathematical functions to describe the R-D behavior of NIC using deep network and statistical modeling.
  arXiv  Detail & Related papers  (2021-06-24T12:23:05Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.