DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement

• URL: http://arxiv.org/abs/2305.08227v1
• Date: Sun, 14 May 2023 19:09:35 GMT
• Title: DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement
• Authors: Hendrik Schr\"oter, Tobias Rosenkranz, Alberto N. Escalante-B., Andreas Maier
• Abstract summary: We present a real-time speech enhancement demo using DeepFilterNet. Our model is able to match state-of-the-art speech enhancement benchmarks while achieving a real-time-factor of 0.19 on a single threaded notebook CPU.
• Score: 10.662665274373387
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multi-frame algorithms for single-channel speech enhancement are able to take advantage from short-time correlations within the speech signal. Deep Filtering (DF) was proposed to directly estimate a complex filter in frequency domain to take advantage of these correlations. In this work, we present a real-time speech enhancement demo using DeepFilterNet. DeepFilterNet's efficiency is enabled by exploiting domain knowledge of speech production and psychoacoustic perception. Our model is able to match state-of-the-art speech enhancement benchmarks while achieving a real-time-factor of 0.19 on a single threaded notebook CPU. The framework as well as pretrained weights have been published under an open source license.

Related papers

• Simple Pooling Front-ends For Efficient Audio Classification [56.59107110017436]
  We show that eliminating the temporal redundancy in the input audio features could be an effective approach for efficient audio classification. We propose a family of simple pooling front-ends (SimPFs) which use simple non-parametric pooling operations to reduce the redundant information. SimPFs can achieve a reduction in more than half of the number of floating point operations for off-the-shelf audio neural networks.
  arXiv  Detail & Related papers  (2022-10-03T14:00:41Z)
• DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio [10.662665274373387]
  DeepFilterNet exploits harmonic structure of speech allowing for efficient speech enhancement (SE) Several optimizations in the training procedure, data augmentation, and network structure result in state-of-the-art SE performance. This makes the algorithm applicable to run on embedded devices in real-time.
  arXiv  Detail & Related papers  (2022-05-11T13:19:41Z)
• End-to-End Neural Audio Coding for Real-Time Communications [22.699018098484707]
  This paper proposes the TFNet, an end-to-end neural audio system with low latency for real-time communications (RTC) An interleaved structure is proposed for temporal filtering to capture both short-term and long-term temporal dependencies. With end-to-end optimization, the TFNet is jointly optimized with speech enhancement and packet loss concealment, yielding a one-for-all network for three tasks.
  arXiv  Detail & Related papers  (2022-01-24T03:06:30Z)
• DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering [9.200520879361916]
  We propose DeepFilterNet, a two stage speech enhancement framework utilizing deep filtering. First, we enhance the spectral envelope using ERB-scaled gains modeling the human frequency perception. The second stage employs deep filtering to enhance the periodic components of speech.
  arXiv  Detail & Related papers  (2021-10-11T20:03:52Z)
• Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models [57.20432226304683]
  Non-autoregressive (NAR) modeling has gained more and more attention in speech processing. We propose a novel end-to-end streaming NAR speech recognition system. We show that the proposed method improves online ASR recognition in low latency conditions.
  arXiv  Detail & Related papers  (2021-07-20T11:42:26Z)
• Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition [64.9317368575585]
  This paper proposes a gated recurrent fusion (GRF) method with joint training framework for robust end-to-end ASR. The GRF algorithm is used to dynamically combine the noisy and enhanced features. The proposed method achieves the relative character error rate (CER) reduction of 10.04% over the conventional joint enhancement and transformer method.
  arXiv  Detail & Related papers  (2020-11-09T08:52:05Z)
• VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition [60.462770498366524]
  We introduce VoiceFilter-Lite, a single-channel source separation model that runs on the device to preserve only the speech signals from a target user. We show that such a model can be quantized as a 8-bit integer model and run in realtime.
  arXiv  Detail & Related papers  (2020-09-09T14:26:56Z)
• DeepVOX: Discovering Features from Raw Audio for Speaker Recognition in Non-ideal Audio Signals [19.053492887246826]
  We propose a deep learning-based technique to deduce the filterbank design from vast amounts of speech audio. The purpose of such a filterbank is to extract features robust to non-ideal audio conditions, such as degraded, short duration, and multi-lingual speech.
  arXiv  Detail & Related papers  (2020-08-26T16:50:26Z)
• Real Time Speech Enhancement in the Waveform Domain [99.02180506016721]
  We present a causal speech enhancement model working on the raw waveform that runs in real-time on a laptop CPU. The proposed model is based on an encoder-decoder architecture with skip-connections. It is capable of removing various kinds of background noise including stationary and non-stationary noises.
  arXiv  Detail & Related papers  (2020-06-23T09:19:13Z)
• End-to-End Automatic Speech Recognition Integrated With CTC-Based Voice Activity Detection [48.80449801938696]
  This paper integrates a voice activity detection function with end-to-end automatic speech recognition. We focus on connectionist temporal classification ( CTC) and its extension ofsynchronous/attention. We use the labels as a cue for detecting speech segments with simple thresholding.
  arXiv  Detail & Related papers  (2020-02-03T03:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.