Cross-utterance ASR Rescoring with Graph-based Label Propagation

• URL: http://arxiv.org/abs/2303.15132v1
• Date: Mon, 27 Mar 2023 12:08:05 GMT
• Title: Cross-utterance ASR Rescoring with Graph-based Label Propagation
• Authors: Srinath Tankasala, Long Chen, Andreas Stolcke, Anirudh Raju, Qianli Deng, Chander Chandak, Aparna Khare, Roland Maas, Venkatesh Ravichandran
• Abstract summary: We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation. In contrast to conventional neural language model (LM) based ASR rescoring/reranking models, our approach focuses on acoustic information.
• Score: 14.669201156515891
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity. In contrast to conventional neural language model (LM) based ASR rescoring/reranking models, our approach focuses on acoustic information and conducts the rescoring collaboratively among utterances, instead of individually. Experiments on the VCTK dataset demonstrate that our approach consistently improves ASR performance, as well as fairness across speaker groups with different accents. Our approach provides a low-cost solution for mitigating the majoritarian bias of ASR systems, without the need to train new domain- or accent-specific models.

Related papers

• Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs [73.74375912785689]
  This paper proposes unified training strategies for speech recognition systems. We demonstrate that training a single model for all three tasks enhances VSR and AVSR performance. We also introduce a greedy pseudo-labelling approach to more effectively leverage unlabelled samples.
  arXiv  Detail & Related papers  (2024-11-04T16:46:53Z)
• Extending Whisper with prompt tuning to target-speaker ASR [18.31992429200396]
  Target-speaker automatic speech recognition (ASR) aims to transcribe the desired speech of a target speaker from overlapped utterances. Most of the existing target-speaker ASR (TS-ASR) methods involve either training from scratch or fully fine-tuning a pre-trained model. This work leverages prompt tuning, a parameter-efficient fine-tuning approach, to extend Whisper, a large-scale single-talker ASR model, to TS-ASR.
  arXiv  Detail & Related papers  (2023-12-13T11:49:16Z)
• Factual Consistency Oriented Speech Recognition [23.754107608608106]
  The proposed framework optimize the ASR model to maximize an expected factual consistency score between ASR hypotheses and ground-truth transcriptions. It is shown that training the ASR models with the proposed framework improves the speech summarization quality as measured by the factual consistency of meeting conversation summaries.
  arXiv  Detail & Related papers  (2023-02-24T00:01:41Z)
• Towards Improved Room Impulse Response Estimation for Speech Recognition [53.04440557465013]
  We propose a novel approach for blind room impulse response (RIR) estimation systems in the context of far-field automatic speech recognition (ASR) We first draw the connection between improved RIR estimation and improved ASR performance, as a means of evaluating neural RIR estimators. We then propose a generative adversarial network (GAN) based architecture that encodes RIR features from reverberant speech and constructs an RIR from the encoded features.
  arXiv  Detail & Related papers  (2022-11-08T00:40:27Z)
• A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings [53.120885867427305]
  Three approaches are evaluated for speaker-attributed automatic speech recognition (SA-ASR) in a meeting scenario. The WD-SOT approach achieves 10.7% relative reduction on averaged speaker-dependent character error rate (SD-CER) The TS-ASR approach also outperforms the FD-SOT approach and brings 16.5% relative average SD-CER reduction.
  arXiv  Detail & Related papers  (2022-03-31T06:39:14Z)
• ASR-Aware End-to-end Neural Diarization [15.172086811068962]
  We present a Conformer-based end-to-end neural diarization (EEND) model that uses both acoustic input and features derived from an automatic speech recognition (ASR) model. Three modifications to the Conformer-based EEND architecture are proposed to incorporate the features. Experiments on the two-speaker English conversations of Switchboard+SRE data sets show that multi-task learning with position-in-word information is the most effective way of utilizing ASR features.
  arXiv  Detail & Related papers  (2022-02-02T21:17:14Z)
• Attention-based Multi-hypothesis Fusion for Speech Summarization [83.04957603852571]
  Speech summarization can be achieved by combining automatic speech recognition (ASR) and text summarization (TS) ASR errors directly affect the quality of the output summary in the cascade approach. We propose a cascade speech summarization model that is robust to ASR errors and that exploits multiple hypotheses generated by ASR to attenuate the effect of ASR errors on the summary.
  arXiv  Detail & Related papers  (2021-11-16T03:00:29Z)
• Neural Model Reprogramming with Similarity Based Mapping for Low-Resource Spoken Command Recognition [71.96870151495536]
  We propose a novel adversarial reprogramming (AR) approach for low-resource spoken command recognition (SCR) The AR procedure aims to modify the acoustic signals (from the target domain) to repurpose a pretrained SCR model. We evaluate the proposed AR-SCR system on three low-resource SCR datasets, including Arabic, Lithuanian, and dysarthric Mandarin speech.
  arXiv  Detail & Related papers  (2021-10-08T05:07:35Z)
• Cross-sentence Neural Language Models for Conversational Speech Recognition [17.317583079824423]
  We propose an effective cross-sentence neural LM approach that reranks the ASR N-best hypotheses of an upcoming sentence. We also explore to extract task-specific global topical information of the cross-sentence history.
  arXiv  Detail & Related papers  (2021-06-13T05:30:16Z)
• Joint Contextual Modeling for ASR Correction and Language Understanding [60.230013453699975]
  We propose multi-task neural approaches to perform contextual language correction on ASR outputs jointly with language understanding (LU) We show that the error rates of off the shelf ASR and following LU systems can be reduced significantly by 14% relative with joint models trained using small amounts of in-domain data.
  arXiv  Detail & Related papers  (2020-01-28T22:09:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.