LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation

• URL: http://arxiv.org/abs/2502.20583v1
• Date: Thu, 27 Feb 2025 22:52:21 GMT
• Title: LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation
• Authors: Keisuke Kamahori, Jungo Kasai, Noriyuki Kojima, Baris Kasikci,
• Abstract summary: We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy.<n> Evaluation results show that our method can compress Whisper large-v3's encoder size by over 50%, matching Whisper medium's size with better transcription accuracy.
• Score: 23.51191930926061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper, rely on deep encoder-decoder architectures, and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy. Our approach leverages the strong low-rank properties observed in intermediate activations: by applying principal component analysis (PCA) with a small calibration dataset, we approximate linear transformations with a chain of low-rank matrix multiplications, and further optimize self-attention to work in the reduced dimension. Evaluation results show that our method can compress Whisper large-v3's encoder size by over 50%, matching Whisper medium's size with better transcription accuracy, thereby establishing a new Pareto-optimal frontier of efficiency and performance. The code of LiteASR is available at https://github.com/efeslab/LiteASR.

Related papers

• Image Coding for Machines via Feature-Preserving Rate-Distortion Optimization [27.97760974010369]
  We show an approach to reduce the effect of compression on a task loss using the distance between features as a distortion metric. We simplify the RDO formulation to make the distortion term computable using block-based encoders. We show up to 10% bit-rate savings for the same computer vision accuracy compared to RDO based on SSE.
  arXiv  Detail & Related papers  (2025-04-03T02:11:26Z)
• Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentation [15.377463849213033]
  EFA is a novel global context modeling mechanism that focuses on functioning the global non-linearity. Our ISR method reduces the key-value resolution at the inference phase, which can mitigate the computation-performance trade-off gap. EDAFormer shows the state-of-the-art performance with the efficient computation compared to the existing transformer-based semantic segmentation models.
  arXiv  Detail & Related papers  (2024-07-24T13:24:25Z)
• Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models [59.57732929473519]
  We apply multiple frame reduction layers in the encoder to compress encoder outputs into a small number of output frames. We demonstrate that we can generate one encoder output frame for every 2.56 sec of input speech, without significantly affecting word error rate on a large-scale voice search task.
  arXiv  Detail & Related papers  (2024-02-27T03:40:44Z)
• Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR [67.63332492134332]
  We design an optimized conformer that is small enough to meet on-device restrictions and has fast inference on TPUs. Our proposed encoder can double as a strong standalone encoder in on device, and as the first part of a high-performance ASR pipeline.
  arXiv  Detail & Related papers  (2023-03-31T23:30:48Z)
• Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Study [25.58608455210458]
  Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech Recognition (ASR) performance in low-resource settings. This article explores different approaches that may be deployed during the fine-tuning to reduce the computations needed in the SSL encoder.
  arXiv  Detail & Related papers  (2023-03-12T19:52:34Z)
• Adding Connectionist Temporal Summarization into Conformer to Improve Its Decoder Efficiency For Speech Recognition [22.61761934996406]
  We propose a novel connectionist temporal summarization (CTS) method that reduces the number of frames required for the attention decoder. With a beamwidth of 4, the LibriSpeech's decoding budget can be reduced by up to 20%. The word error rate (WER) is reduced by 6% relative at the beam width of 1 and by 3% relative at the beam width of 4.
  arXiv  Detail & Related papers  (2022-04-08T07:24:00Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Relaxed Attention: A Simple Method to Boost Performance of End-to-End Automatic Speech Recognition [27.530537066239116]
  We introduce the concept of relaxed attention, which is a gradual injection of a uniform distribution to the encoder-decoder attention weights during training. We find that transformers trained with relaxed attention outperform the standard baseline models consistently during decoding with external language models. On WSJ, we set a new benchmark for transformer-based end-to-end speech recognition with a word error rate of 3.65%, outperforming state of the art (4.20%) by 13.1% relative.
  arXiv  Detail & Related papers  (2021-07-02T21:01:17Z)
• Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers [56.56220390953412]
  We extend our prior work by introducing the Conformer architecture to further improve the accuracy. We demonstrate that the extended Transformer provides state-of-the-art end-to-end ASR performance.
  arXiv  Detail & Related papers  (2021-04-19T16:18:00Z)
• Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning with Self-Knowledge Distillation [11.52842516726486]
  We propose a Transformer-based ASR model with the time reduction layer, in which we incorporate time reduction layer inside transformer encoder layers. We also introduce a fine-tuning approach for pre-trained ASR models using self-knowledge distillation (S-KD) which further improves the performance of our ASR model. With language model (LM) fusion, we achieve new state-of-the-art word error rate (WER) results for Transformer-based ASR models.
  arXiv  Detail & Related papers  (2021-03-17T21:02:36Z)
• Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input [54.82369261350497]
  We propose a CTC-enhanced NAR transformer, which generates target sequence by refining predictions of the CTC module. Experimental results show that our method outperforms all previous NAR counterparts and achieves 50x faster decoding speed than a strong AR baseline with only 0.0 0.3 absolute CER degradation on Aishell-1 and Aishell-2 datasets.
  arXiv  Detail & Related papers  (2020-10-28T15:00:09Z)
• Streaming automatic speech recognition with the transformer model [59.58318952000571]
  We propose a transformer based end-to-end ASR system for streaming ASR. We apply time-restricted self-attention for the encoder and triggered attention for the encoder-decoder attention mechanism. Our proposed streaming transformer architecture achieves 2.8% and 7.2% WER for the "clean" and "other" test data of LibriSpeech.
  arXiv  Detail & Related papers  (2020-01-08T18:58:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.