High-precision Voice Search Query Correction via Retrievable Speech-text Embedings

• URL: http://arxiv.org/abs/2401.04235v1
• Date: Mon, 8 Jan 2024 20:59:56 GMT
• Title: High-precision Voice Search Query Correction via Retrievable Speech-text Embedings
• Abstract summary: Speech recognition systems can suffer from poor recall for various reasons, such as noisy audio, lack of sufficient training data, etc. Previous work has shown that recall can be improved by retrieving rewrite candidates from a large database of likely, contextually-relevant alternatives to the hypothesis text. We show a relative word error rate (WER) reduction of 6% on utterances whose transcripts appear in the candidate set, without increasing WER on general utterances.
• Score: 13.4015768521113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic speech recognition (ASR) systems can suffer from poor recall for various reasons, such as noisy audio, lack of sufficient training data, etc. Previous work has shown that recall can be improved by retrieving rewrite candidates from a large database of likely, contextually-relevant alternatives to the hypothesis text using nearest-neighbors search over embeddings of the ASR hypothesis text to correct and candidate corrections. However, ASR-hypothesis-based retrieval can yield poor precision if the textual hypotheses are too phonetically dissimilar to the transcript truth. In this paper, we eliminate the hypothesis-audio mismatch problem by querying the correction database directly using embeddings derived from the utterance audio; the embeddings of the utterance audio and candidate corrections are produced by multimodal speech-text embedding networks trained to place the embedding of the audio of an utterance and the embedding of its corresponding textual transcript close together. After locating an appropriate correction candidate using nearest-neighbor search, we score the candidate with its speech-text embedding distance before adding the candidate to the original n-best list. We show a relative word error rate (WER) reduction of 6% on utterances whose transcripts appear in the candidate set, without increasing WER on general utterances.

Related papers

• Few-Shot Contrastive Adaptation for Audio Abuse Detection in Low-Resource Indic Languages [3.5238606794194816]
  Abusive speech detection is becoming increasingly important as social media shifts towards voice-based interaction.<n>We investigate whether Contrastive Language-Audio Pre-training can support abusive speech detection directly from audio.
  arXiv  Detail & Related papers  (2026-04-10T08:23:03Z)
• Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction [7.787275515915814]
  Recent or infrequent movie titles may not be sufficiently represented in the E2E ASR system's training data.<n>We propose a phonetic correction system that consists of a phonetic search based on the ASR model's output.<n>We find that our approach improves word error rate between 4.4 and 7.6% relative on benchmarks of popular movie titles.
  arXiv  Detail & Related papers  (2025-06-06T14:25:18Z)
• Prompting Whisper for Improved Verbatim Transcription and End-to-end Miscue Detection [7.650371454756065]
  We propose an end-to-end architecture that incorporates the target reading text via prompting and is trained for both improved verbatim transcription and direct miscue detection.<n>We conducted two case studies -- children's read-aloud and adult atypical speech -- and found that our proposed strategies improve verbatim transcription and miscue detection compared to current state-of-the-art.
  arXiv  Detail & Related papers  (2025-05-29T16:34:47Z)
• Detecting the Undetectable: Assessing the Efficacy of Current Spoof Detection Methods Against Seamless Speech Edits [82.8859060022651]
  We introduce the Speech INfilling Edit (SINE) dataset, created with Voicebox. Subjective evaluations confirm that speech edited using this novel technique is more challenging to detect than conventional cut-and-paste methods. Despite human difficulty, experimental results demonstrate that self-supervised-based detectors can achieve remarkable performance in detection, localization, and generalization.
  arXiv  Detail & Related papers  (2025-01-07T14:17:47Z)
• Speech Retrieval-Augmented Generation without Automatic Speech Recognition [4.731446054087683]
  SpeechRAG is a novel framework designed for open-question answering over spoken data. Our proposed approach fine-tunes a pre-trained speech encoder into a speech adapter fed into a frozen large language model. By aligning the embedding spaces of text and speech, our speech retriever directly retrieves audio passages from text-based queries.
  arXiv  Detail & Related papers  (2024-12-21T06:16:04Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Looking and Listening: Audio Guided Text Recognition [62.98768236858089]
  Text recognition in the wild is a long-standing problem in computer vision. Recent studies suggest vision and language processing are effective for scene text recognition. Yet, solving edit errors such as add, delete, or replace is still the main challenge for existing approaches. We propose the AudioOCR, a simple yet effective probabilistic audio decoder for mel spectrogram sequence prediction.
  arXiv  Detail & Related papers  (2023-06-06T08:08:18Z)
• SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappings [76.87664008338317]
  Contextual spelling correction models are an alternative to shallow fusion to improve automatic speech recognition. We propose a novel algorithm for candidate retrieval based on misspelled n-gram mappings. Experiments on Spoken Wikipedia show 21.4% word error rate improvement compared to a baseline ASR system.
  arXiv  Detail & Related papers  (2023-06-04T10:00:12Z)
• On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answering [13.013751306590303]
  We study the robustness of lexical and dense retrievers against questions with synthetic ASR noise. We create a new dataset with questions voiced by human users and use their transcriptions to show that the retrieval performance can further degrade when dealing with natural ASR noise instead of synthetic ASR noise.
  arXiv  Detail & Related papers  (2022-09-26T18:29:36Z)
• Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information [55.75018546938499]
  We propose the speaker embedding-aware neural diarization (SEND) method, which predicts the power set encoded labels. Our method achieves lower diarization error rate than the target-speaker voice activity detection.
  arXiv  Detail & Related papers  (2021-11-28T12:51:04Z)
• Integrated Semantic and Phonetic Post-correction for Chinese Speech Recognition [1.2914521751805657]
  We propose a novel approach to collectively exploit the contextualized representation and the phonetic information between the error and its replacing candidates to alleviate the error rate of Chinese ASR. Our experiment results on real world speech recognition showed that our proposed method has evidently lower than the baseline model.
  arXiv  Detail & Related papers  (2021-11-16T11:55:27Z)
• FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition [92.12910821300034]
  We propose FastCorrect 2, an error correction model that takes multiple ASR candidates as input for better correction accuracy. FastCorrect 2 achieves better performance than the cascaded re-scoring and correction pipeline.
  arXiv  Detail & Related papers  (2021-09-29T13:48:03Z)
• Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration [62.75234183218897]
  We propose a one-stage context-aware framework to generate natural and coherent target speech without any training data of the speaker. We generate the mel-spectrogram of the edited speech with a transformer-based decoder. It outperforms a recent zero-shot TTS engine by a large margin.
  arXiv  Detail & Related papers  (2021-09-12T04:17:53Z)
• Personalized Keyphrase Detection using Speaker and Environment Information [24.766475943042202]
  We introduce a streaming keyphrase detection system that can be easily customized to accurately detect any phrase composed of words from a large vocabulary. The system is implemented with an end-to-end trained automatic speech recognition (ASR) model and a text-independent speaker verification model.
  arXiv  Detail & Related papers  (2021-04-28T18:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.