Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting

• URL: http://arxiv.org/abs/2211.06478v1
• Date: Fri, 11 Nov 2022 20:41:46 GMT
• Title: Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting
• Abstract summary: We present a novel approach to adapt a sequence-to-sequence Transformer-Transducer ASR system to the keyword spotting (KWS) task. We achieve this by replacing the keyword in the text transcription with a special token kw> and training the system to detect the kw> token in an audio stream. We find that our approach significantly outperforms ASR based KWS systems.
• Score: 15.61189030504559
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a novel approach to adapt a sequence-to-sequence Transformer-Transducer ASR system to the keyword spotting (KWS) task. We achieve this by replacing the keyword in the text transcription with a special token <kw> and training the system to detect the <kw> token in an audio stream. At inference time, we create a decision function inspired by conventional KWS approaches, to make our approach more suitable for the KWS task. Furthermore, we introduce a specific keyword spotting loss by adapting the sequence-discriminative Minimum Bayes-Risk training technique. We find that our approach significantly outperforms ASR based KWS systems. When compared with a conventional keyword spotting system, our proposal has similar performance while bringing the advantages and flexibility of sequence-to-sequence training. Additionally, when combined with the conventional KWS system, our approach can improve the performance at any operation point.

Related papers

• PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Framework [17.616287091537128]
  We introduce the Prompt Phrases Prediction Network (PPN), an encoder-decoder architecture designed to effectively extract keyword prompts embeddings.<n>We employ the PPN encoder to encode the keyword prompts and infuse the prompt embedding into the Prompt-guided KWS encoder by utilizing a Prompt-acoustic Multi-head Cross-attention (MHCA)<n>Experiments show that PromptKWS improves the wakeup rate by over 10% compared to baseline system.
  arXiv  Detail & Related papers  (2026-08-11T12:10:22Z)
• MASSFormer: Mobility-Aware Spectrum Sensing using Transformer-Driven Tiered Structure [3.6194127685460553]
  We develop a mobility-aware transformer-driven structure (MASSFormer) based cooperative sensing method. Our method considers a dynamic scenario involving mobile primary users (PUs) and secondary users (SUs) The proposed method is tested under imperfect reporting channel scenarios to show robustness.
  arXiv  Detail & Related papers  (2024-09-26T05:25:25Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Condition-Invariant Semantic Segmentation [77.10045325743644]
  We implement Condition-Invariant Semantic (CISS) on the current state-of-the-art domain adaptation architecture. Our method achieves the second-best performance on the normal-to-adverse Cityscapes$to$ACDC benchmark. CISS is shown to generalize well to domains unseen during training, such as BDD100K-night and ACDC-night.
  arXiv  Detail & Related papers  (2023-05-27T03:05:07Z)
• Adaptive Multi-Agent Continuous Learning System [1.2752808844888015]
  We propose an adaptive multi-agent clustering recognition system that can be self-supervised driven. The system is designed to use some different functional agents to build up a connection structure to improve adaptability to cope with environmental diverse demands.
  arXiv  Detail & Related papers  (2022-12-15T07:39:50Z)
• Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers [6.744385328015561]
  Task-oriented dialogue systems, such as Apple Siri and Amazon Alexa, require a semantic parsing module in order to process user utterances and understand the action to be performed. This semantic parsing component was initially implemented by rule-based or statistical slot-filling approaches for processing simple queries. In this article, we advance the research on neural-reduce semantic parsing for task-oriented dialogue.
  arXiv  Detail & Related papers  (2022-10-21T14:19:47Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• BBS-KWS:The Mandarin Keyword Spotting System Won the Video Keyword Wakeup Challenge [14.312411161892705]
  This paper introduces the system submitted by the Yidun NISP team to the video keyword wakeup challenge. We propose a mandarin keyword spotting system (KWS) with several novel and effective improvements. In the VKW task, the BBS-KWS system achieves significant gains over the baseline and won the first place in two tracks.
  arXiv  Detail & Related papers  (2021-12-03T07:27:13Z)
• Multi-task Learning with Cross Attention for Keyword Spotting [8.103605110339519]
  Keywords spotting (KWS) is an important technique for speech applications, which enables users to activate devices by speaking a keyword phrase. There is a mismatch between the training criterion (phoneme recognition) and the target task (KWS) Recently, multi-task learning has been applied to KWS to exploit both ASR and KWS training data.
  arXiv  Detail & Related papers  (2021-07-15T22:38:16Z)
• Keyphrase Extraction with Dynamic Graph Convolutional Networks and Diversified Inference [50.768682650658384]
  Keyphrase extraction (KE) aims to summarize a set of phrases that accurately express a concept or a topic covered in a given document. Recent Sequence-to-Sequence (Seq2Seq) based generative framework is widely used in KE task, and it has obtained competitive performance on various benchmarks. In this paper, we propose to adopt the Dynamic Graph Convolutional Networks (DGCN) to solve the above two problems simultaneously.
  arXiv  Detail & Related papers  (2020-10-24T08:11:23Z)
• Cross-Thought for Sentence Encoder Pre-training [89.32270059777025]
  Cross-Thought is a novel approach to pre-training sequence encoder. We train a Transformer-based sequence encoder over a large set of short sequences. Experiments on question answering and textual entailment tasks demonstrate that our pre-trained encoder can outperform state-of-the-art encoders.
  arXiv  Detail & Related papers  (2020-10-07T21:02:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.