Learning Task-Based Trainable Neuromorphic ADCs via Power-Aware Distillation

• URL: http://arxiv.org/abs/2409.15300v1
• Date: Wed, 4 Sep 2024 12:10:47 GMT
• Title: Learning Task-Based Trainable Neuromorphic ADCs via Power-Aware Distillation
• Abstract summary: We introduce a data-driven algorithm that tunes task-based memristive ADCs alongside both digital and analog processing. The proposed approach is shown to enhance accuracy by up to 27% and reduce power consumption by up to 66%. Even under noisy conditions, our method achieves substantial gains, with accuracy improvements of up to 19% and power reductions of up to 57%.
• Score: 24.103729832774935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability to process signals in digital form depends on analog-to-digital converters (ADCs). Traditionally, ADCs are designed to ensure that the digital representation closely matches the analog signal. However, recent studies have shown that significant power and memory savings can be achieved through task-based acquisition, where the acquisition process is tailored to the downstream processing task. An emerging technology for task-based acquisition involves the use of memristors, which are considered key enablers for neuromorphic computing. Memristors can implement ADCs with tunable mappings, allowing adaptation to specific system tasks or power constraints. In this work, we study task-based acquisition for a generic classification task using memristive ADCs. We consider the unique characteristics of this such neuromorphic ADCs, including their power consumption and noisy read-write behavior, and propose a physically compliant model based on resistive successive approximation register ADCs integrated with memristor components, enabling the adjustment of quantization regions. To optimize performance, we introduce a data-driven algorithm that jointly tunes task-based memristive ADCs alongside both digital and analog processing. Our design addresses the inherent stochasticity of memristors through power-aware distillation, complemented by a specialized learning algorithm that adapts to their unique analog-to-digital mapping. The proposed approach is shown to enhance accuracy by up to 27% and reduce power consumption by up to 66% compared to uniform ADCs. Even under noisy conditions, our method achieves substantial gains, with accuracy improvements of up to 19% and power reductions of up to 57%. These results highlight the effectiveness of our power-aware neuromorphic ADCs in improving system performance across diverse tasks.

Related papers

• Co-Optimization of Analog Kolmogorov-Arnold Networks for Low-Power Function Approximation in Flexible Electronics [1.4864449542438336]
  This work introduces Analog Kolmogorov-Arnold Networks (AKANs), developed via hardware-software co-optimization.<n>AKANs approximate complex multivariate functions accurately under hardware imperfections.<n>Results show up to 55% area and 50% power savings, with average reductions of nearly 30% across datasets.
  arXiv  Detail & Related papers  (2026-06-26T09:35:48Z)
• A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs [5.6496088684920345]
  ReRAM-based accelerators process neural networks via analog Computing-in-Memory (CiM) for ultra-high energy efficiency. This work explores the hardware implementation of the Sigmoid and SoftMax activation functions of neural networks with crossbarally binarized neurons. We propose a complete ReRAM-based Analog Computing Accelerator (RACA) that accelerates neural network computation by leveraging inferenceally binarized neurons.
  arXiv  Detail & Related papers  (2024-12-27T09:38:19Z)
• Synergistic Development of Perovskite Memristors and Algorithms for Robust Analog Computing [53.77822620185878]
  We propose a synergistic methodology to concurrently optimize perovskite memristor fabrication and develop robust analog DNNs. We develop "BayesMulti", a training strategy utilizing BO-guided noise injection to improve the resistance of analog DNNs to memristor imperfections. Our integrated approach enables use of analog computing in much deeper and wider networks, achieving up to 100-fold improvements.
  arXiv  Detail & Related papers  (2024-12-03T19:20:08Z)
• Towards training digitally-tied analog blocks via hybrid gradient computation [1.800676987432211]
  We introduce Feedforward-tied Energy-based Models (ff-EBMs) We derive a novel algorithm to compute gradients end-to-end in ff-EBMs by backpropagating and "eq-propagating" through feedforward and energy-based parts respectively. Our approach offers a principled, scalable, and incremental roadmap to gradually integrate self-trainable analog computational primitives into existing digital accelerators.
  arXiv  Detail & Related papers  (2024-09-05T07:22:19Z)
• Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
  Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries. By digitizing data throughout the product life cycle, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures. mechanisms that leverage sensing Industrial Internet of Things (IIoT) devices to share data for the construction of DTs are susceptible to adverse selection problems.
  arXiv  Detail & Related papers  (2024-08-02T10:47:10Z)
• StoX-Net: Stochastic Processing of Partial Sums for Efficient In-Memory Computing DNN Accelerators [5.245727758971415]
  Crossbarsoftware-based in-memory computing (IMC) has emerged as a promising platform for hardware acceleration of deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2024-07-17T07:56:43Z)
• Pruning for Improved ADC Efficiency in Crossbar-based Analog In-memory Accelerators [9.169425049927554]
  Crossbar-based analog in-memory architectures are attractive for acceleration of deep neural networks (DNN) They require analog-to-digital converters (ADCs) to communicate crossbar outputs. ADCs consume a significant portion of energy and area of every crossbar processing unit. We motivate crossbar-attuned pruning to target ADC-specific inefficiencies.
  arXiv  Detail & Related papers  (2024-03-19T18:26:45Z)
• EKGNet: A 10.96{\mu}W Fully Analog Neural Network for Intra-Patient Arrhythmia Classification [79.7946379395238]
  We present an integrated approach by combining analog computing and deep learning for electrocardiogram (ECG) arrhythmia classification. We propose EKGNet, a hardware-efficient and fully analog arrhythmia classification architecture that archives high accuracy with low power consumption.
  arXiv  Detail & Related papers  (2023-10-24T02:37:49Z)
• DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection [49.196182908826565]
  Auditory Attention Detection (AAD) aims to detect target speaker from brain signals in a multi-speaker environment. Current approaches primarily rely on traditional convolutional neural network designed for processing Euclidean data like images. This paper proposes a dynamical graph self-distillation (DGSD) approach for AAD, which does not require speech stimuli as input.
  arXiv  Detail & Related papers  (2023-09-07T13:43:46Z)
• ADC/DAC-Free Analog Acceleration of Deep Neural Networks with Frequency Transformation [2.7488316163114823]
  This paper proposes a novel approach to an energy-efficient acceleration of frequency-domain neural networks by utilizing analog-domain frequency-based tensor transformations. Our approach achieves more compact cells by eliminating the need for trainable parameters in the transformation matrix. On a 16$times$16 crossbars, for 8-bit input processing, the proposed approach achieves the energy efficiency of 1602 tera operations per second per Watt.
  arXiv  Detail & Related papers  (2023-09-04T19:19:39Z)
• Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning [57.83232242068982]
  Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. It remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. This work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy.
  arXiv  Detail & Related papers  (2023-05-25T15:46:20Z)
• An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification [97.28167655721766]
  We propose a novel doubly accelerated gradient descent (ADSGD) method for sparsity regularized loss minimization problems. We first prove that ADSGD can achieve a linear convergence rate and lower overall computational complexity.
  arXiv  Detail & Related papers  (2022-08-11T22:27:22Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning [1.4502611532302039]
  An innovative intelligent control strategy for buck DC-DC converter with constant power loads (CPLs) is constructed for the first time. A Markov Decision Process (MDP) model and the deep Q network (DQN) algorithm are defined for DC-DC converter. A model-free based deep reinforcement learning (DRL) control strategy is appropriately designed to adjust the agent-environment interaction.
  arXiv  Detail & Related papers  (2020-08-11T06:38:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.