Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions

• URL: http://arxiv.org/abs/2402.13076v2
• Date: Wed, 26 Feb 2025 05:33:32 GMT
• Title: Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions
• Abstract summary: We find that the influence of weight parameters on power consumption varies depending on factors such as invocation frequency and memory allocation.<n>We propose design principles that enhance on-device speech recognition models by reducing power consumption with minimal impact on accuracy.
• Score: 20.180037657388763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Power consumption plays a crucial role in on-device streaming speech recognition, significantly influencing the user experience. This study explores how the configuration of weight parameters in speech recognition models affects their overall energy efficiency. We found that the influence of these parameters on power consumption varies depending on factors such as invocation frequency and memory allocation. Leveraging these insights, we propose design principles that enhance on-device speech recognition models by reducing power consumption with minimal impact on accuracy. Our approach, which adjusts model components based on their specific energy sensitivities, achieves up to 47% lower energy usage while preserving comparable model accuracy and improving real-time performance compared to leading methods.

Related papers

• Power Consumption Patterns Using Telemetry Data [8.702932756755043]
  This paper examines the analysis of package power consumption using Intel's telemetry data.<n>It challenges the prevailing belief that hardware choice is the primary determinant of a device's power consumption.
  arXiv  Detail & Related papers  (2026-02-25T19:04:57Z)
• Optimising for Energy Efficiency and Performance in Machine Learning [3.8803432012641395]
  We show that Energy Consumption Optimiser (ECOpt) optimises for energy efficiency and model performance.<n>ECOpt quantifies the trade-off between these metrics as an interpretable frontier.<n>We show that ECOpt can have a net positive environmental impact and use it to uncover seven models for CIFAR-10 that improve upon the state of the art.
  arXiv  Detail & Related papers  (2026-01-13T21:28:58Z)
• Comparing energy consumption and accuracy in text classification inference [0.9208007322096533]
  This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference.<n>The best-performing model in terms of accuracy can also be energy-efficient, while larger LLMs tend to consume significantly more energy with lower classification accuracy.
  arXiv  Detail & Related papers  (2025-08-19T18:00:08Z)
• Energy Efficiency in AI for 5G and Beyond: A DeepRx Case Study [0.276240219662896]
  We evaluate the energy consumption of DeepRX, a deep learning receiver based on a fully convolutional ResNet architecture.<n>Key contribution is the application of knowledge distillation to train a compact DeepRX student model that emulates the performance of the teacher model but with reduced energy consumption.<n>Performance is measured by comparing the Bit Error Rate (BER) performance versus Signal-to-Interference & Noise Ratio (SINR) values of the distilled model and a model trained from scratch.
  arXiv  Detail & Related papers  (2025-07-14T15:54:06Z)
• Diffused Responsibility: Analyzing the Energy Consumption of Generative Text-to-Audio Diffusion Models [15.100667215184036]
  We conduct an analysis of the energy usage of 7 state-of-the-art text-to-audio diffusion-based generative models.<n>We also aim to identify an optimal balance between audio quality and energy consumption.
  arXiv  Detail & Related papers  (2025-05-12T14:36:47Z)
• Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models [55.46269953415811]
  We identify ToM-sensitive parameters and show that perturbing as little as 0.001% of these parameters significantly degrades ToM performance. Our results have implications for enhancing model alignment, mitigating biases, and improving AI systems designed for human interaction.
  arXiv  Detail & Related papers  (2025-04-05T17:45:42Z)
• Investigating Energy Efficiency and Performance Trade-offs in LLM Inference Across Tasks and DVFS Settings [1.5749416770494706]
  Large language models (LLMs) have shown significant improvements in many natural language processing (NLP) tasks. LLMs are resource-intensive, requiring extensive computational resources both during training and inference. As their adoption accelerates, the sustainability of LLMs has become a critical issue.
  arXiv  Detail & Related papers  (2025-01-14T16:02:33Z)
• Energy-Aware Dynamic Neural Inference [39.04688735618206]
  We introduce an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers.
  arXiv  Detail & Related papers  (2024-11-04T16:51:22Z)
• Impact of ML Optimization Tactics on Greener Pre-Trained ML Models [46.78148962732881]
  This study aims to (i) analyze image classification datasets and pre-trained models, (ii) improve inference efficiency by comparing optimized and non-optimized models, and (iii) assess the economic impact of the optimizations. We conduct a controlled experiment to evaluate the impact of various PyTorch optimization techniques (dynamic quantization, torch.compile, local pruning, and global pruning) to 42 Hugging Face models for image classification. Dynamic quantization demonstrates significant reductions in inference time and energy consumption, making it highly suitable for large-scale systems.
  arXiv  Detail & Related papers  (2024-09-19T16:23:03Z)
• Normalizing Energy Consumption for Hardware-Independent Evaluation [9.658615045493734]
  We present a novel methodology for normalizing energy consumption across different hardware platforms. Our approach shows that the number of reference points, the type of regression and the inclusion of computational metrics significantly influences the normalization process.
  arXiv  Detail & Related papers  (2024-09-09T13:38:00Z)
• Deep Learning Models in Speech Recognition: Measuring GPU Energy Consumption, Impact of Noise and Model Quantization for Edge Deployment [0.0]
  This study examines the effects of quantization, memory demands, and energy consumption on the performance of various ASR model inference on the NVIDIA Jetson Orin Nano. We found that changing precision from fp32 to fp16 halves the energy consumption for audio transcription across different models, with minimal performance degradation.
  arXiv  Detail & Related papers  (2024-05-02T05:09:07Z)
• Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models [109.06052781040916]
  We introduce a technique to enhance the inference efficiency of parameter-shared language models. We also propose a simple pre-training technique that leads to fully or partially shared models. Results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs.
  arXiv  Detail & Related papers  (2023-10-19T15:13:58Z)
• Scaling Laws for Sparsely-Connected Foundation Models [70.41266138010657]
  We explore the impact of parameter sparsity on the scaling behavior of Transformers trained on massive datasets. We identify the first scaling law describing the relationship between weight sparsity, number of non-zero parameters, and amount of training data.
  arXiv  Detail & Related papers  (2023-09-15T16:29:27Z)
• Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition [19.772585241974138]
  Streaming speech recognition models usually process a limited number of tokens each time. bottleneck lies in the linear projection layers of multi-head attention and feedforward networks. We propose folding attention, a technique targeting these linear layers, significantly reducing model size and improving memory and power efficiency.
  arXiv  Detail & Related papers  (2023-09-14T19:01:08Z)
• PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement [41.872384434583466]
  We propose a learning objective that formalizes differences in perceptual quality. We identify temporal acoustic parameters that are non-differentiable. We develop a neural network estimator that can accurately predict their time-series values.
  arXiv  Detail & Related papers  (2023-02-16T05:17:06Z)
• TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement [41.872384434583466]
  We provide a differentiable estimator for four categories of low-level acoustic descriptors involving: frequency-related parameters, energy or amplitude-related parameters, spectral balance parameters, and temporal features. We show that adding TAP as an auxiliary objective in speech enhancement produces speech with improved perceptual quality and intelligibility.
  arXiv  Detail & Related papers  (2023-02-16T04:57:11Z)
• LEAF + AIO: Edge-Assisted Energy-Aware Object Detection for Mobile Augmented Reality [77.00418462388525]
  Mobile augmented reality (MAR) applications are significantly energy-guzzling. We design an edge-based energy-aware MAR system that enables MAR devices to dynamically change their configurations. Our proposed dynamic MAR configuration adaptations can minimize the per frame energy consumption of multiple MAR clients.
  arXiv  Detail & Related papers  (2022-05-27T06:11:50Z)
• Powerpropagation: A sparsity inducing weight reparameterisation [65.85142037667065]
  We introduce Powerpropagation, a new weight- parameterisation for neural networks that leads to inherently sparse models. Models trained in this manner exhibit similar performance, but have a distribution with markedly higher density at zero, allowing more parameters to be pruned safely. Here, we combine Powerpropagation with a traditional weight-pruning technique as well as recent state-of-the-art sparse-to-sparse algorithms, showing superior performance on the ImageNet benchmark.
  arXiv  Detail & Related papers  (2021-10-01T10:03:57Z)
• Time-domain Speech Enhancement with Generative Adversarial Learning [53.74228907273269]
  This paper proposes a new framework called Time-domain Speech Enhancement Generative Adversarial Network (TSEGAN) TSEGAN is an extension of the generative adversarial network (GAN) in time-domain with metric evaluation to mitigate the scaling problem. In addition, we provide a new method based on objective function mapping for the theoretical analysis of the performance of Metric GAN.
  arXiv  Detail & Related papers  (2021-03-30T08:09:49Z)
• On Dynamic Noise Influence in Differentially Private Learning [102.6791870228147]
  Private Gradient Descent (PGD) is a commonly used private learning framework, which noises based on the Differential protocol. Recent studies show that emphdynamic privacy schedules can improve at the final iteration, yet yet theoreticals of the effectiveness of such schedules remain limited. This paper provides comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
  arXiv  Detail & Related papers  (2021-01-19T02:04:00Z)
• Efficient End-to-End Speech Recognition Using Performers in Conformers [74.71219757585841]
  We propose to reduce the complexity of model architectures in addition to model sizes. The proposed model yields competitive performance on the LibriSpeech corpus with 10 millions of parameters and linear complexity.
  arXiv  Detail & Related papers  (2020-11-09T05:22:57Z)
• AdaSense: Adaptive Low-Power Sensing and Activity Recognition for Wearable Devices [2.0119495998780925]
  AdaSense is a sensing, feature extraction and classification co-optimized framework for Human Activity Recognition. The framework selects configurations that represent the pareto-frontier of the accuracy and energy trade-off. The introduced approach achieves 69% reduction in the power consumption of the sensor with less than 1.5% decrease in the activity recognition accuracy.
  arXiv  Detail & Related papers  (2020-06-10T15:17:11Z)
• Audio Impairment Recognition Using a Correlation-Based Feature Representation [85.08880949780894]
  We propose a new representation of hand-crafted features that is based on the correlation of feature pairs. We show superior performance in terms of compact feature dimensionality and improved computational speed in the test stage.
  arXiv  Detail & Related papers  (2020-03-22T13:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.