Speech Boosting: Low-Latency Live Speech Enhancement for TWS Earbuds

• URL: http://arxiv.org/abs/2409.18705v1
• Date: Fri, 27 Sep 2024 12:47:36 GMT
• Title: Speech Boosting: Low-Latency Live Speech Enhancement for TWS Earbuds
• Authors: Hanbin Bae, Pavel Andreev, Azat Saginbaev, Nicholas Babaev, Won-Jun Lee, Hosang Sung, Hoon-Young Cho,
• Abstract summary: This paper introduces a speech enhancement solution tailored for true wireless stereo (TWS) earbuds on-device usage. The solution was specifically designed to support conversations in noisy environments, with active noise cancellation (ANC) activated.
• Score: 7.360661203298394
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper introduces a speech enhancement solution tailored for true wireless stereo (TWS) earbuds on-device usage. The solution was specifically designed to support conversations in noisy environments, with active noise cancellation (ANC) activated. The primary challenges for speech enhancement models in this context arise from computational complexity that limits on-device usage and latency that must be less than 3 ms to preserve a live conversation. To address these issues, we evaluated several crucial design elements, including the network architecture and domain, design of loss functions, pruning method, and hardware-specific optimization. Consequently, we demonstrated substantial improvements in speech enhancement quality compared with that in baseline models, while simultaneously reducing the computational complexity and algorithmic latency.

Related papers

• VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment [101.2489492032816]
  VALL-E R is a robust and efficient zero-shot Text-to-Speech system. This research has the potential to be applied to meaningful projects, including the creation of speech for those affected by aphasia.
  arXiv  Detail & Related papers  (2024-06-12T04:09:44Z)
• Robust Active Speaker Detection in Noisy Environments [29.785749048315616]
  We formulate a robust active speaker detection (rASD) problem in noisy environments. Existing ASD approaches leverage both audio and visual modalities, but non-speech sounds in the surrounding environment can negatively impact performance. We propose a novel framework that utilizes audio-visual speech separation as guidance to learn noise-free audio features.
  arXiv  Detail & Related papers  (2024-03-27T20:52:30Z)
• CheapNET: Improving Light-weight speech enhancement network by projected loss function [0.8192907805418583]
  We introduce a novel projection loss function, diverging from MSE, to enhance noise suppression. For echo cancellation, the function enables direct predictions on LAEC pre-processed outputs. Our noise suppression model achieves near state-of-the-art results with only 3.1M parameters and 0.4GFlops/s computational load.
  arXiv  Detail & Related papers  (2023-11-27T16:03:42Z)
• Speech-enhanced and Noise-aware Networks for Robust Speech Recognition [25.279902171523233]
  A noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition. The two proposed systems achieve word error rate (WER) of 3.90% and 3.55%, respectively, on the Aurora-4 task. Compared with the best existing systems that use bigram and trigram language models for decoding, the proposed CNN-TDNNF-based system achieves a relative WER reduction of 15.20% and 33.53%, respectively.
  arXiv  Detail & Related papers  (2022-03-25T15:04:51Z)
• A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning [57.28467469709369]
  We investigate on designing a compact audio-visual wake word spotting (WWS) system by utilizing visual information. We introduce a neural network pruning strategy via the lottery ticket hypothesis in an iterative fine-tuning manner (LTH-IF) The proposed audio-visual system achieves significant performance improvements over the single-modality (audio-only or video-only) system under different noisy conditions.
  arXiv  Detail & Related papers  (2022-02-17T08:26:25Z)
• Implicit Acoustic Echo Cancellation for Keyword Spotting and Device-Directed Speech Detection [2.7393821783237184]
  In many speech-enabled human-machine interaction scenarios, user speech can overlap with the device playback audio. We propose an implicit acoustic echo cancellation framework where a neural network is trained to exploit the additional information from a reference microphone channel. We show a $56%$ reduction in false-reject rate for the DDD task during device playback conditions.
  arXiv  Detail & Related papers  (2021-11-20T17:21:16Z)
• Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction [109.44933866397123]
  Noise robustness is essential for deploying automatic speech recognition systems in real-world environments. We employ a noise-robust representation learned by a refined self-supervised framework for noisy speech recognition. We achieve comparable performance to the best supervised approach reported with only 16% of labeled data.
  arXiv  Detail & Related papers  (2021-10-28T20:39:02Z)
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition [60.462770498366524]
  We introduce VoiceFilter-Lite, a single-channel source separation model that runs on the device to preserve only the speech signals from a target user. We show that such a model can be quantized as a 8-bit integer model and run in realtime.
  arXiv  Detail & Related papers  (2020-09-09T14:26:56Z)
• TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices [71.68436132514542]
  We introduce the concept of attention condensers for building low-footprint, highly-efficient deep neural networks for on-device speech recognition on the edge. To illustrate its efficacy, we introduce TinySpeech, low-precision deep neural networks tailored for on-device speech recognition.
  arXiv  Detail & Related papers  (2020-08-10T16:34:52Z)
• Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances [53.063441357826484]
  Speaker recognition systems based on deep speaker embeddings have achieved significant performance in controlled conditions. Speaker verification on short utterances in uncontrolled noisy environment conditions is one of the most challenging and highly demanded tasks. This paper presents approaches aimed to achieve two goals: a) improve the quality of far-field speaker verification systems in the presence of environmental noise, reverberation and b) reduce the system qualitydegradation for short utterances.
  arXiv  Detail & Related papers  (2020-02-14T13:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.