Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation

• URL: http://arxiv.org/abs/2105.06598v1
• Date: Fri, 14 May 2021 00:41:42 GMT
• Title: Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation
• Abstract summary: We present a unified and hardware efficient architecture for two stage voice trigger detection (VTD) and false trigger mitigation (FTM) tasks. Traditional FTM systems rely on automatic speech recognition lattices which are computationally expensive to obtain on device. We propose a streaming transformer architecture, which progressively processes incoming audio chunks and maintains audio context to perform both VTD and FTM tasks.
• Score: 9.691823786336716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a unified and hardware efficient architecture for two stage voice trigger detection (VTD) and false trigger mitigation (FTM) tasks. Two stage VTD systems of voice assistants can get falsely activated to audio segments acoustically similar to the trigger phrase of interest. FTM systems cancel such activations by using post trigger audio context. Traditional FTM systems rely on automatic speech recognition lattices which are computationally expensive to obtain on device. We propose a streaming transformer (TF) encoder architecture, which progressively processes incoming audio chunks and maintains audio context to perform both VTD and FTM tasks using only acoustic features. The proposed joint model yields an average 18% relative reduction in false reject rate (FRR) for the VTD task at a given false alarm rate. Moreover, our model suppresses 95% of the false triggers with an additional one second of post-trigger audio. Finally, on-device measurements show 32% reduction in runtime memory and 56% reduction in inference time compared to non-streaming version of the model.

Related papers

• Zero-Shot Temporal Localisation of Audio Deepfakes in Multi-Speaker Conversations [4.152774504513197]
  Training-free five-stage pipeline wraps a frozen binary detector and adds segment-level output with no retraining.<n>On 180 constructed multi-speaker conversations from ASVspoof 5, the system attains temporal intersection-over-union 0.90, temporal detection rate 0.95, and MS-DCF 0.26 with a strong backbone.<n>Its false-alarm rate on genuine speech is below 6%, falling under 2% on genuine real multi-speaker dialogue (AMI)<n>Under an identical pipeline, a trained localiser improves temporal IoU by only about 0.04, bounding the cost of forgoing supervision.
  arXiv  Detail & Related papers  (2026-09-09T11:27:26Z)
• TRADE: Transducer-Augmented Decoder for Speech LLM [52.69726809996728]
  Speech Large Language Models (Speech LLMs) lack a principled mechanism for streaming inference.<n>We propose TRADE TRansducer-Augmented DEcoder, which augments a multimodal LLM with a transducer branch that shares the audio encoder.<n>Three design choices make the system accurate, streamable, and long-form capable.
  arXiv  Detail & Related papers  (2026-06-07T07:15:34Z)
• MOSS-TTS Technical Report [62.0595749527453]
  MOSS-TTS is a speech generation foundation model built on a scalable recipe.<n>It compresses 24 kHz audio to 12.5 fps with variable-bitrate RVQ and unified semantic-acoustic representations.<n>It supports zero-shot voice cloning, token-level duration control, phoneme-/pinyin-level pronunciation control, smooth code-switching, and stable long-form generation.
  arXiv  Detail & Related papers  (2026-03-18T09:08:06Z)
• Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine [16.046905753937384]
  This work introduces an efficient ACoM method that can compress and quantize any chosen intermediate feature representation of an already trained speech/audio downstream model.<n>Our approach employs task-specific loss guidance alongside residual vector quantization (RVQ) losses, providing ultra-low codecs (i.e., less than 200 bps) with a minimal loss of the downstream model performance.
  arXiv  Detail & Related papers  (2025-07-17T00:32:07Z)
• Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition [1.0690007351232649]
  We propose a layer-adapted fusion (LAF) model, called Qifusion-Net, which does not require any prior knowledge about the target accent. Experiment results demonstrate that our proposed methods outperform the baseline with relative reductions of 22.1$%$ and 17.2$%$ in character error rate (CER) across multi accent test datasets.
  arXiv  Detail & Related papers  (2024-07-03T11:35:52Z)
• Autoregressive Diffusion Transformer for Text-to-Speech Synthesis [39.32761051774537]
  We propose encoding audio as vector sequences in continuous space $mathbb Rd$ and autoregressively generating these sequences. High-bitrate continuous speech representation enables almost flawless reconstruction, allowing our model to achieve nearly perfect speech editing.
  arXiv  Detail & Related papers  (2024-06-08T18:57:13Z)
• Multimodal Data and Resource Efficient Device-Directed Speech Detection with Large Foundation Models [43.155061160275196]
  We explore the possibility of making interactions with virtual assistants more natural by eliminating the need for a trigger phrase. Our goal is to determine whether a user addressed the virtual assistant based on signals obtained from the streaming audio recorded by the device microphone. We address this task by combining 1-best hypotheses and decoder signals from an automatic speech recognition system with acoustic representations from an audio encoder.
  arXiv  Detail & Related papers  (2023-12-06T17:29:03Z)
• Efficient Audio Captioning Transformer with Patchout and Text Guidance [74.59739661383726]
  We propose a full Transformer architecture that utilizes Patchout as proposed in [1], significantly reducing the computational complexity and avoiding overfitting. The caption generation is partly conditioned on textual AudioSet tags extracted by a pre-trained classification model. Our proposed method received the Judges Award at the Task6A of DCASE Challenge 2022.
  arXiv  Detail & Related papers  (2023-04-06T07:58:27Z)
• Fully Automated End-to-End Fake Audio Detection [57.78459588263812]
  This paper proposes a fully automated end-toend fake audio detection method. We first use wav2vec pre-trained model to obtain a high-level representation of the speech. For the network structure, we use a modified version of the differentiable architecture search (DARTS) named light-DARTS.
  arXiv  Detail & Related papers  (2022-08-20T06:46:55Z)
• ETAD: A Unified Framework for Efficient Temporal Action Detection [70.21104995731085]
  Untrimmed video understanding such as temporal action detection (TAD) often suffers from the pain of huge demand for computing resources. We build a unified framework for efficient end-to-end temporal action detection (ETAD) ETAD achieves state-of-the-art performance on both THUMOS-14 and ActivityNet-1.3.
  arXiv  Detail & Related papers  (2022-05-14T21:16:21Z)
• WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition [59.975078145303605]
  We propose a novel framework, namely WNARS, using hybrid CTC-attention AED models and weighted finite-state transducers. On the AISHELL-1 task, our WNARS achieves a character error rate of 5.22% with 640ms latency, to the best of our knowledge, which is the state-of-the-art performance for online ASR.
  arXiv  Detail & Related papers  (2021-04-08T07:56:03Z)
• Knowledge Transfer for Efficient On-device False Trigger Mitigation [17.53768388104929]
  An undirected utterance is termed as a "false trigger" and false trigger mitigation (FTM) is essential for designing a privacy-centric smart assistant. We propose an LSTM-based FTM architecture which determines the user intent from acoustic features directly without explicitly generating ASR transcripts.
  arXiv  Detail & Related papers  (2020-10-20T20:01:44Z)
• Audio-visual Multi-channel Recognition of Overlapped Speech [79.21950701506732]
  This paper presents an audio-visual multi-channel overlapped speech recognition system featuring tightly integrated separation front-end and recognition back-end. Experiments suggest that the proposed multi-channel AVSR system outperforms the baseline audio-only ASR system by up to 6.81% (26.83% relative) and 22.22% (56.87% relative) absolute word error rate (WER) reduction on overlapped speech constructed using either simulation or replaying of the lipreading sentence 2 dataset respectively.
  arXiv  Detail & Related papers  (2020-05-18T10:31:19Z)
• High Performance Sequence-to-Sequence Model for Streaming Speech Recognition [19.488757267198498]
  Sequence-to-sequence models have started to achieve state-of-the-art performance on standard speech recognition tasks. But when it comes to performing run-on recognition on an input stream of audio data, these models face several challenges. We introduce an additional loss function controlling the uncertainty of the attention mechanism, a modified beam search identifying partial, stable hypotheses, ways of working with BLSTM in the encoder, and the use of chunked BLSTM.
  arXiv  Detail & Related papers  (2020-03-22T23:04:32Z)
• Streaming automatic speech recognition with the transformer model [59.58318952000571]
  We propose a transformer based end-to-end ASR system for streaming ASR. We apply time-restricted self-attention for the encoder and triggered attention for the encoder-decoder attention mechanism. Our proposed streaming transformer architecture achieves 2.8% and 7.2% WER for the "clean" and "other" test data of LibriSpeech.
  arXiv  Detail & Related papers  (2020-01-08T18:58:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.