Reducing ADC Front-end Costs During Training of On-sensor Printed Multilayer Perceptrons

• URL: http://arxiv.org/abs/2411.08674v2
• Date: Thu, 14 Nov 2024 10:28:31 GMT
• Title: Reducing ADC Front-end Costs During Training of On-sensor Printed Multilayer Perceptrons
• Authors: Florentia Afentaki, Paula Carolina Lozano Duarte, Georgios Zervakis, Mehdi B. Tahoori,
• Abstract summary: Printed electronics technology offers a cost-and fully-customizable solution to computational needs beyond traditional silicon technologies. Low-resolution fabrication of printedelectronics poses a challenge for integrating complex designs like those of machine learn-ing (ML) classification systems.
• Score: 0.9406794506458744
• License:
• Abstract: Printed electronics technology offers a cost-effectiveand fully-customizable solution to computational needs beyondthe capabilities of traditional silicon technologies, offering ad-vantages such as on-demand manufacturing and conformal, low-cost hardware. However, the low-resolution fabrication of printedelectronics, which results in large feature sizes, poses a challengefor integrating complex designs like those of machine learn-ing (ML) classification systems. Current literature optimizes onlythe Multilayer Perceptron (MLP) circuit within the classificationsystem, while the cost of analog-to-digital converters (ADCs)is overlooked. Printed applications frequently require on-sensorprocessing, yet while the digital classifier has been extensivelyoptimized, the analog-to-digital interfacing, specifically the ADCs,dominates the total area and energy consumption. In this work,we target digital printed MLP classifiers and we propose thedesign of customized ADCs per MLP's input which involvesminimizing the distinct represented numbers for each input,simplifying thus the ADC's circuitry. Incorporating this ADCoptimization in the MLP training, enables eliminating ADC levelsand the respective comparators, while still maintaining highclassification accuracy. Our approach achieves 11.2x lower ADCarea for less than 5% accuracy drop across varying MLPs.

Related papers

• Progressive Mixed-Precision Decoding for Efficient LLM Inference [49.05448842542558]
  We introduce Progressive Mixed-Precision Decoding (PMPD) to address the memory-boundedness of decoding. PMPD achieves 1.4$-$12.2$times$ speedup in matrix-vector multiplications over fp16 models. Our approach delivers a throughput gain of 3.8$-$8.0$times$ over fp16 models and up to 1.54$times$ over uniform quantization approaches.
  arXiv  Detail & Related papers  (2024-10-17T11:46:33Z)
• Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
  Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries. By digitizing data throughout the product life cycle, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures. mechanisms that leverage sensing Industrial Internet of Things (IIoT) devices to share data for the construction of DTs are susceptible to adverse selection problems.
  arXiv  Detail & Related papers  (2024-08-02T10:47:10Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Embedding Hardware Approximations in Discrete Genetic-based Training for Printed MLPs [1.4694098707981968]
  Printed Electronics (PE) enables stretchable, conformal,and non-toxic hardware. PE are constrained by larger feature sizes, making it challenging to implement complex circuits such as machine learning (ML)aware circuits. In this paper, we maximize the benefits of approximate computing by integrating hardware approximation into the training process.
  arXiv  Detail & Related papers  (2024-02-05T11:52:23Z)
• Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons [0.8274768545559366]
  Printed Electronics (PE) offer unparalleled features such as non-recurring engineering costs, ultra-low manufacturing costs, and on-demand fabrication. PE face certain limitations due to their large feature sizes, that impede the realization of complex circuits. We propose an automated framework for designing ultra-low power Multilayer Perceptron (MLP) classifiers.
  arXiv  Detail & Related papers  (2023-12-29T14:16:11Z)
• Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference [13.000030080938078]
  computational cost of transformer models makes them inefficient in low-latency or low-power applications. We introduce the Adaptive Computation Module (ACM), a generic module that dynamically adapts its computational load to match the estimated difficulty of the input on a per-token basis. Our evaluation of transformer models in computer vision and speech recognition demonstrates that substituting layers with ACMs significantly reduces inference costs without degrading the downstream accuracy for a wide interval of user-defined budgets.
  arXiv  Detail & Related papers  (2023-12-15T20:39:43Z)
• On-sensor Printed Machine Learning Classification via Bespoke ADC and Decision Tree Co-Design [3.919502921806021]
  Printed electronics (PE) technology provides cost-effective hardware with unmet customization, due to their low non-recurring engineering and fabrication costs. PE exhibit features such as flexibility, stretchability, porosity, and conformality, which make them a prominent candidate for enabling ubiquitous computing. We propose the design of fully customized ADCs and present, for the first time, a co-design framework for generating bespoke Decision Tree classifiers.
  arXiv  Detail & Related papers  (2023-12-02T16:28:09Z)
• Co-Design of Approximate Multilayer Perceptron for Ultra-Resource Constrained Printed Circuits [4.865819809855699]
  Large feature sizes in Printed Electronics (PE) prohibit the realization of complex printed machine learning circuits. We present, for the first time, an automated printed-aware software/hardware co-design framework that exploits approximate computing principles to enable ultra-resource constrained printed multilayer perceptrons (MLPs) Our evaluation demonstrates that, compared to the state-of-the-art baseline, our circuits feature on average 6x (5.7x) lower area (power) and less than 1% accuracy loss.
  arXiv  Detail & Related papers  (2023-02-28T13:55:19Z)
• Unifying Synergies between Self-supervised Learning and Dynamic Computation [53.66628188936682]
  We present a novel perspective on the interplay between SSL and DC paradigms. We show that it is feasible to simultaneously learn a dense and gated sub-network from scratch in a SSL setting. The co-evolution during pre-training of both dense and gated encoder offers a good accuracy-efficiency trade-off.
  arXiv  Detail & Related papers  (2023-01-22T17:12:58Z)
• The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers [59.87030906486969]
  This paper studies the curious phenomenon for machine learning models with Transformer architectures that their activation maps are sparse. We show that sparsity is a prevalent phenomenon that occurs for both natural language processing and vision tasks. We discuss how sparsity immediately implies a way to significantly reduce the FLOP count and improve efficiency for Transformers.
  arXiv  Detail & Related papers  (2022-10-12T15:25:19Z)
• Single-Shot Optical Neural Network [55.41644538483948]
  'Weight-stationary' analog optical and electronic hardware has been proposed to reduce the compute resources required by deep neural networks. We present a scalable, single-shot-per-layer weight-stationary optical processor.
  arXiv  Detail & Related papers  (2022-05-18T17:49:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.