Open-vocabulary Keyword-spotting with Adaptive Instance Normalization

• URL: http://arxiv.org/abs/2309.08561v1
• Date: Wed, 13 Sep 2023 13:49:42 GMT
• Title: Open-vocabulary Keyword-spotting with Adaptive Instance Normalization
• Authors: Aviv Navon, Aviv Shamsian, Neta Glazer, Gill Hetz, Joseph Keshet
• Abstract summary: We propose AdaKWS, a novel method for keyword spotting in which a text encoder is trained to output keyword-conditioned normalization parameters. We show significant improvements over recent keyword spotting and ASR baselines.
• Score: 18.250276540068047
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open vocabulary keyword spotting is a crucial and challenging task in automatic speech recognition (ASR) that focuses on detecting user-defined keywords within a spoken utterance. Keyword spotting methods commonly map the audio utterance and keyword into a joint embedding space to obtain some affinity score. In this work, we propose AdaKWS, a novel method for keyword spotting in which a text encoder is trained to output keyword-conditioned normalization parameters. These parameters are used to process the auditory input. We provide an extensive evaluation using challenging and diverse multi-lingual benchmarks and show significant improvements over recent keyword spotting and ASR baselines. Furthermore, we study the effectiveness of our approach on low-resource languages that were unseen during the training. The results demonstrate a substantial performance improvement compared to baseline methods.

Related papers

• An Analysis of BPE Vocabulary Trimming in Neural Machine Translation [56.383793805299234]
  vocabulary trimming is a postprocessing step that replaces rare subwords with their component subwords. We show that vocabulary trimming fails to improve performance and is even prone to incurring heavy degradation.
  arXiv  Detail & Related papers  (2024-03-30T15:29:49Z)
• To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement [58.96644066571205]
  We show that existing deep keyword spotting mechanisms can be improved by Successive Refinement. We show across multiple models with size ranging from 13K parameters to 2.41M parameters, the successive refinement technique reduces FA by up to a factor of 8. Our proposed approach is "plug-and-play" and can be applied to any deep keyword spotting model.
  arXiv  Detail & Related papers  (2023-04-06T23:49:29Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [103.6153593636399]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning) It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting [23.627625026135505]
  We propose a novel end-to-end user-defined keyword spotting method. Our method compares input queries with an enrolled text keyword sequence. We introduce the LibriPhrase dataset for efficiently training keyword spotting models.
  arXiv  Detail & Related papers  (2022-06-30T16:40:31Z)
• Short-Term Word-Learning in a Dynamically Changing Environment [63.025297637716534]
  We show how to supplement an end-to-end ASR system with a word/phrase memory and a mechanism to access this memory to recognize the words and phrases correctly. We demonstrate significant improvements in the detection rate of new words with only a minor increase in false alarms.
  arXiv  Detail & Related papers  (2022-03-29T10:05:39Z)
• Representation Learning for Resource-Constrained Keyphrase Generation [78.02577815973764]
  We introduce salient span recovery and salient span prediction as guided denoising language modeling objectives. We show the effectiveness of the proposed approach for low-resource and zero-shot keyphrase generation.
  arXiv  Detail & Related papers  (2022-03-15T17:48:04Z)
• Spell my name: keyword boosted speech recognition [25.931897154065663]
  uncommon words such as names and technical terminology are important to understanding conversations in context. We propose a simple but powerful ASR decoding method that can better recognise these uncommon keywords. The method boosts the probabilities of given keywords in a beam search based on acoustic model predictions. We demonstrate the effectiveness of our method on the LibriSpeeech test sets and also internal data of real-world conversations.
  arXiv  Detail & Related papers  (2021-10-06T14:16:57Z)
• Teaching keyword spotters to spot new keywords with limited examples [6.251896411370577]
  We present KeySEM, a speech embedding model pre-trained on the task of recognizing a large number of keywords. KeySEM is well suited to on-device environments where post-deployment learning and ease of customization are often desirable.
  arXiv  Detail & Related papers  (2021-06-04T12:43:36Z)
• Seeing wake words: Audio-visual Keyword Spotting [103.12655603634337]
  KWS-Net is a novel convolutional architecture that uses a similarity map intermediate representation to separate the task into sequence matching and pattern detection. We show that our method generalises to other languages, specifically French and German, and achieves a comparable performance to English with less language specific data.
  arXiv  Detail & Related papers  (2020-09-02T17:57:38Z)
• Few-Shot Keyword Spotting With Prototypical Networks [3.6930948691311016]
  keyword spotting has been widely used in many voice interfaces such as Amazon's Alexa and Google Home. We first formulate this problem as a few-shot keyword spotting and approach it using metric learning. We then propose a solution to the prototypical few-shot keyword spotting problem using temporal and dilated convolutions on networks.
  arXiv  Detail & Related papers  (2020-07-25T20:17:56Z)
• Keyword-Attentive Deep Semantic Matching [1.8416014644193064]
  We propose a keyword-attentive approach to improve deep semantic matching. We first leverage domain tags from a large corpus to generate a domain-enhanced keyword dictionary. During model training, we propose a new negative sampling approach based on keyword coverage between the input pair.
  arXiv  Detail & Related papers  (2020-03-11T10:18:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.