Mixture Encoder for Joint Speech Separation and Recognition

• URL: http://arxiv.org/abs/2306.12173v1
• Date: Wed, 21 Jun 2023 11:01:31 GMT
• Title: Mixture Encoder for Joint Speech Separation and Recognition
• Authors: Simon Berger, Peter Vieting, Christoph Boeddeker, Ralf Schl\"uter and Reinhold Haeb-Umbach
• Abstract summary: Multi-speaker automatic speech recognition is crucial for many real-world applications. Existing approaches can be divided into modular and end-to-end methods. End-to-end models process overlapped speech directly in a single, powerful neural network.
• Score: 15.13598115379631
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-speaker automatic speech recognition (ASR) is crucial for many real-world applications, but it requires dedicated modeling techniques. Existing approaches can be divided into modular and end-to-end methods. Modular approaches separate speakers and recognize each of them with a single-speaker ASR system. End-to-end models process overlapped speech directly in a single, powerful neural network. This work proposes a middle-ground approach that leverages explicit speech separation similarly to the modular approach but also incorporates mixture speech information directly into the ASR module in order to mitigate the propagation of errors made by the speech separator. We also explore a way to exchange cross-speaker context information through a layer that combines information of the individual speakers. Our system is optimized through separate and joint training stages and achieves a relative improvement of 7% in word error rate over a purely modular setup on the SMS-WSJ task.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.