Enhanced ASR Robustness to Packet Loss with a Front-End Adaptation Network

• URL: http://arxiv.org/abs/2406.18928v1
• Date: Thu, 27 Jun 2024 06:40:01 GMT
• Title: Enhanced ASR Robustness to Packet Loss with a Front-End Adaptation Network
• Authors: Yehoshua Dissen, Shiry Yonash, Israel Cohen, Joseph Keshet,
• Abstract summary: This study is focused on recovering from packet loss to improve the word error rate (WER) of ASR models. We propose using a front-end adaptation network connected to a frozen ASR model. Experiments demonstrate that the adaptation network, trained on Whisper's criteria, notably reduces word error rates across domains and languages.
• Score: 23.034147003704483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the realm of automatic speech recognition (ASR), robustness in noisy environments remains a significant challenge. Recent ASR models, such as Whisper, have shown promise, but their efficacy in noisy conditions can be further enhanced. This study is focused on recovering from packet loss to improve the word error rate (WER) of ASR models. We propose using a front-end adaptation network connected to a frozen ASR model. The adaptation network is trained to modify the corrupted input spectrum by minimizing the criteria of the ASR model in addition to an enhancement loss function. Our experiments demonstrate that the adaptation network, trained on Whisper's criteria, notably reduces word error rates across domains and languages in packet-loss scenarios. This improvement is achieved with minimal affect to Whisper model's foundational performance, underscoring our method's practicality and potential in enhancing ASR models in challenging acoustic environments.

Related papers

• Transferable Adversarial Attacks against ASR [43.766547483367795]
  We study the vulnerability of practical black-box attacks in cutting-edge automatic speech recognition models. We propose a speech-aware gradient optimization approach (SAGO) for ASR, which forces mistranscription with minimal impact on human imperceptibility. Our comprehensive experimental results reveal performance enhancements compared to baseline approaches across five models on two databases.
  arXiv  Detail & Related papers  (2024-11-14T06:32:31Z)
• Crossmodal ASR Error Correction with Discrete Speech Units [16.58209270191005]
  We propose a post-ASR processing approach for ASR Error Correction (AEC) We explore pre-training and fine-tuning strategies and uncover an ASR domain discrepancy phenomenon. We propose the incorporation of discrete speech units to align with and enhance the word embeddings for improving AEC quality.
  arXiv  Detail & Related papers  (2024-05-26T19:58:38Z)
• Speech enhancement with frequency domain auto-regressive modeling [34.55703785405481]
  Speech applications in far-field real world settings often deal with signals that are corrupted by reverberation. We propose a unified framework of speech dereverberation for improving the speech quality and the automatic speech recognition (ASR) performance.
  arXiv  Detail & Related papers  (2023-09-24T03:25:51Z)
• Towards Improved Room Impulse Response Estimation for Speech Recognition [53.04440557465013]
  We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of far-field automatic speech recognition (ASR) We first draw the connection between improved RIR estimation and improved ASR performance, as a means of evaluating neural RIR estimators. We then propose a generative adversarial network (GAN) based architecture that encodes RIR features from reverberant speech and constructs an RIR from the encoded features.
  arXiv  Detail & Related papers  (2022-11-08T00:40:27Z)
• Calibrate and Refine! A Novel and Agile Framework for ASR-error Robust Intent Detection [8.842878491315124]
  We propose a novel framework called CR-ID for ASR error robust intent detection with two plug-and-play modules. Experimental results on SNIPS dataset show that, our proposed CR-ID framework achieves competitive performance.
  arXiv  Detail & Related papers  (2022-05-23T02:54:11Z)
• Attention-based Multi-hypothesis Fusion for Speech Summarization [83.04957603852571]
  Speech summarization can be achieved by combining automatic speech recognition (ASR) and text summarization (TS) ASR errors directly affect the quality of the output summary in the cascade approach. We propose a cascade speech summarization model that is robust to ASR errors and that exploits multiple hypotheses generated by ASR to attenuate the effect of ASR errors on the summary.
  arXiv  Detail & Related papers  (2021-11-16T03:00:29Z)
• Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction [109.44933866397123]
  Noise robustness is essential for deploying automatic speech recognition systems in real-world environments. We employ a noise-robust representation learned by a refined self-supervised framework for noisy speech recognition. We achieve comparable performance to the best supervised approach reported with only 16% of labeled data.
  arXiv  Detail & Related papers  (2021-10-28T20:39:02Z)
• Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networks [10.723935272906461]
  Adversarial training of end-to-end (E2E) ASR systems using generative adversarial networks (GAN) has recently been explored. We introduce a novel framework for fine-tuning a pre-trained ASR model using the GAN objective. Our proposed approach outperforms baselines and conventional GAN-based adversarial models.
  arXiv  Detail & Related papers  (2021-03-10T17:40:48Z)
• Deep Adaptive Inference Networks for Single Image Super-Resolution [72.7304455761067]
  Single image super-resolution (SISR) has witnessed tremendous progress in recent years owing to the deployment of deep convolutional neural networks (CNNs) In this paper, we take a step forward to address this issue by leveraging the adaptive inference networks for deep SISR (AdaDSR) Our AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth.
  arXiv  Detail & Related papers  (2020-04-08T10:08:20Z)
• Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement [102.48582597586233]
  We present a U-Net based attention model, U-Net$_At$, to enhance adversarial speech signals. We conduct experiments on the automatic speech recognition (ASR) task with adversarial audio attacks.
  arXiv  Detail & Related papers  (2020-03-31T02:16:34Z)
• Improving noise robust automatic speech recognition with single-channel time-domain enhancement network [100.1041336974175]
  We show that a single-channel time-domain denoising approach can significantly improve ASR performance. We show that single-channel noise reduction can still improve ASR performance.
  arXiv  Detail & Related papers  (2020-03-09T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.