Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction

• URL: http://arxiv.org/abs/2506.06117v1
• Date: Fri, 06 Jun 2025 14:25:18 GMT
• Title: Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction
• Abstract summary: Recent or infrequent movie titles may not be sufficiently represented in the E2E ASR system's training data.<n>We propose a phonetic correction system that consists of a phonetic search based on the ASR model's output.<n>We find that our approach improves word error rate between 4.4 and 7.6% relative on benchmarks of popular movie titles.
• Score: 7.787275515915814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end (E2E) Automatic Speech Recognition (ASR) models are trained using paired audio-text samples that are expensive to obtain, since high-quality ground-truth data requires human annotators. Voice search applications, such as digital media players, leverage ASR to allow users to search by voice as opposed to an on-screen keyboard. However, recent or infrequent movie titles may not be sufficiently represented in the E2E ASR system's training data, and hence, may suffer poor recognition. In this paper, we propose a phonetic correction system that consists of (a) a phonetic search based on the ASR model's output that generates phonetic alternatives that may not be considered by the E2E system, and (b) a rescorer component that combines the ASR model recognition and the phonetic alternatives, and select a final system output. We find that our approach improves word error rate between 4.4 and 7.6% relative on benchmarks of popular movie titles over a series of competitive baselines.

Related papers

• ATIR: Towards Audio-Text Interleaved Contextual Retrieval [63.68521448682396]
  We introduce the Audio-Text Interleaved contextual Retrieval (ATIR) task, where queries can alternate between audio and text modalities.<n>This benchmark substantially addresses the limitations of existing audio retrieval datasets in semantic retrieval.
  arXiv  Detail & Related papers  (2026-04-22T07:11:58Z)
• AHELM: A Holistic Evaluation of Audio-Language Models [78.20477815156484]
  multimodal audio-language models (ALMs) take interleaved audio and text as input and output text.<n>AHELM is a benchmark that aggregates various datasets -- including 2 new synthetic audio-text datasets called PARADE and CoRe-Bench.<n>We also standardize the prompts, inference parameters, and evaluation metrics to ensure equitable comparisons across models.
  arXiv  Detail & Related papers  (2025-08-29T07:40:39Z)
• Listening and Seeing Again: Generative Error Correction for Audio-Visual Speech Recognition [39.206005299985605]
  We propose a novel GER paradigm for AVSR, termed AVGER, that follows the concept of listening and seeing again''<n>The proposed AVGER can reduce Word Error Rate (WER) by 24% compared to current mainstream AVSR systems.
  arXiv  Detail & Related papers  (2025-01-03T10:51:14Z)
• Enhancing Audiovisual Speech Recognition through Bifocal Preference Optimization [59.1277150358203]
  We propose using a preference optimization strategy to improve speech recognition accuracy for real-world videos.<n>First, we create preference data via simulating common errors that occurred in AV-ASR from two focals.<n>Second, we propose BPO-AVASR, a Bifocal Preference Optimization method to improve AV-ASR models by leveraging both input-side and output-side preference.
  arXiv  Detail & Related papers  (2024-12-26T00:26:45Z)
• High-precision Voice Search Query Correction via Retrievable Speech-text Embedings [13.4015768521113]
  Speech recognition systems can suffer from poor recall for various reasons, such as noisy audio, lack of sufficient training data, etc. Previous work has shown that recall can be improved by retrieving rewrite candidates from a large database of likely, contextually-relevant alternatives to the hypothesis text. We show a relative word error rate (WER) reduction of 6% on utterances whose transcripts appear in the candidate set, without increasing WER on general utterances.
  arXiv  Detail & Related papers  (2024-01-08T20:59:56Z)
• Modality Confidence Aware Training for Robust End-to-End Spoken Language Understanding [18.616202196061966]
  End-to-end (E2E) spoken language understanding (SLU) systems that generate a semantic parse from speech have become more promising recently. This approach uses a single model that utilizes audio and text representations from pre-trained speech recognition models (ASR) We propose a novel E2E SLU system that enhances robustness to ASR errors by fusing audio and text representations based on the estimated modality confidence of ASR hypotheses.
  arXiv  Detail & Related papers  (2023-07-22T17:47:31Z)
• AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations [88.30635799280923]
  We introduce AV-data2vec which builds audio-visual representations based on predicting contextualized representations. Results on LRS3 show that AV-data2vec consistently outperforms existing methods with the same amount of data and model size.
  arXiv  Detail & Related papers  (2023-02-10T02:55:52Z)
• ESB: A Benchmark For Multi-Domain End-to-End Speech Recognition [100.30565531246165]
  Speech recognition systems require dataset-specific tuning. This tuning requirement can lead to systems failing to generalise to other datasets and domains. We introduce the End-to-end Speech Benchmark (ESB) for evaluating the performance of a single automatic speech recognition system.
  arXiv  Detail & Related papers  (2022-10-24T15:58:48Z)
• End-to-end contextual asr based on posterior distribution adaptation for hybrid ctc/attention system [61.148549738631814]
  End-to-end (E2E) speech recognition architectures assemble all components of traditional speech recognition system into a single model. Although it simplifies ASR system, it introduces contextual ASR drawback: the E2E model has worse performance on utterances containing infrequent proper nouns. We propose to add a contextual bias attention (CBA) module to attention based encoder decoder (AED) model to improve its ability of recognizing the contextual phrases.
  arXiv  Detail & Related papers  (2022-02-18T03:26:02Z)
• FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition [92.12910821300034]
  We propose FastCorrect 2, an error correction model that takes multiple ASR candidates as input for better correction accuracy. FastCorrect 2 achieves better performance than the cascaded re-scoring and correction pipeline.
  arXiv  Detail & Related papers  (2021-09-29T13:48:03Z)
• Mondegreen: A Post-Processing Solution to Speech Recognition Error Correction for Voice Search Queries [2.2962622992492143]
  We introduce an approach, Mondegreen, to correct voice queries in text space without depending on audio signals. We present an analysis showing how different the language distribution coming from user voice queries is from that in traditional text corpora. We then demonstrate that Mondegreen can achieve significant improvements in increased user interaction by correcting user voice queries in one of the largest search systems in Google.
  arXiv  Detail & Related papers  (2021-05-20T17:45:46Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)
• An End-to-End Mispronunciation Detection System for L2 English Speech Leveraging Novel Anti-Phone Modeling [11.894724235336872]
  Mispronunciation detection and diagnosis (MDD) is a core component of computer-assisted pronunciation training (CAPT) We propose to conduct MDD with a novel end- to-end automatic speech recognition (E2E-based ASR) approach. In particular, we expand the original L2 phone set with their corresponding anti-phone set, aiming to provide better mispronunciation detection and diagnosis feedback.
  arXiv  Detail & Related papers  (2020-05-25T07:27:47Z)
• Deliberation Model Based Two-Pass End-to-End Speech Recognition [52.45841282906516]
  A two-pass model has been proposed to rescore streamed hypotheses using the non-streaming Listen, Attend and Spell (LAS) model. The model attends to acoustics to rescore hypotheses, as opposed to a class of neural correction models that use only first-pass text hypotheses. A bidirectional encoder is used to extract context information from first-pass hypotheses.
  arXiv  Detail & Related papers  (2020-03-17T22:01:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.