Adaptive Knowledge Distillation for Device-Directed Speech Detection

• URL: http://arxiv.org/abs/2508.02801v1
• Date: Mon, 04 Aug 2025 18:12:28 GMT
• Title: Adaptive Knowledge Distillation for Device-Directed Speech Detection
• Authors: Hyung Gun Chi, Florian Pesce, Wonil Chang, Oggi Rudovic, Arturo Argueta, Stefan Braun, Vineet Garg, Ahmed Hussen Abdelaziz,
• Abstract summary: We introduce a novel adaptive KD method that transfers knowledge from general representations of an large pre-trained acoustic encoder (teacher)<n>We demonstrate that the proposed adaptive KD outperforms the student model without distillation in the keyword invocations, with an improvement of +26% and +19% in terms of Equal Error Rate.
• Score: 5.521554644415849
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Device-directed speech detection (DDSD) is a binary classification task that separates the user's queries to a voice assistant (VA) from background speech or side conversations. This is important for achieving naturalistic user experience. To this end, we propose knowledge distillation (KD) to enhance DDSD accuracy while ensuring efficient deployment. Specifically, we introduce a novel adaptive KD method that transfers knowledge from general representations of an ASR large pre-trained acoustic encoder (teacher). We apply task-specific adapters, on top of the (frozen) teacher encoder, trained jointly with the student model on DDSD. We demonstrate that the proposed adaptive KD outperforms the student model without distillation in the keyword and keyword-free (follow-up) invocations, with an improvement of +26% and +19% in terms of Equal Error Rate, respectively. We also show that this approach generalizes across the transformer and conformer-based model architectures.

Related papers

• Mutual Learning for Acoustic Matching and Dereverberation via Visual Scene-driven Diffusion [93.32354378820648]
  We introduce MVSD, a mutual learning framework based on diffusion models. MVSD considers the two tasks symmetrically, exploiting the reciprocal relationship to facilitate learning from inverse tasks. Our framework can improve the performance of the reverberator and dereverberator.
  arXiv  Detail & Related papers  (2024-07-15T00:47:56Z)
• DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding [51.32965203977845]
  We propose the use of discrete speech units (DSU) instead of continuous-valued speech encoder outputs. The proposed model shows robust performance on speech inputs from seen/unseen domains and instruction-following capability in spoken question answering. Our findings suggest that the ASR task and datasets are not crucial in instruction-tuning for spoken question answering tasks.
  arXiv  Detail & Related papers  (2024-06-13T17:28:13Z)
• Part Representation Learning with Teacher-Student Decoder for Occluded Person Re-identification [65.63180725319906]
  We propose a Teacher-Student Decoder (TSD) framework for occluded person ReID. Our proposed TSD consists of a Parsing-aware Teacher Decoder (PTD) and a Standard Student Decoder (SSD)
  arXiv  Detail & Related papers  (2023-12-15T13:54:48Z)
• Modality Dropout for Multimodal Device Directed Speech Detection using Verbal and Non-Verbal Features [11.212228410835435]
  We study the use of non-verbal cues, specifically prosody features, in addition to verbal cues for device-directed speech detection (DDSD) We present different approaches to combine scores and embeddings from prosody with the corresponding verbal cues, finding that prosody improves performance by upto 8.5% in terms of false acceptance rate (FA) Our use of modality dropout techniques improves the performance of these models by 7.4% in terms of FA when evaluated with missing modalities during inference time.
  arXiv  Detail & Related papers  (2023-10-23T18:09:31Z)
• An Effective Transformer-based Contextual Model and Temporal Gate Pooling for Speaker Identification [0.0]
  This paper introduces an effective end-to-end speaker identification model applied Transformer-based contextual model. We propose a pooling method, Temporal Gate Pooling, with powerful learning ability for speaker identification. The proposed method has achieved an accuracy of 87.1% with 28.5M parameters, demonstrating comparable precision to wav2vec2 with 317.7M parameters.
  arXiv  Detail & Related papers  (2023-08-22T07:34:07Z)
• Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation [66.25738680429463]
  Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model. We propose inconsistent knowledge distillation (IKD) which aims to distill knowledge inherent in the teacher model's counter-intuitive perceptions. Our method outperforms state-of-the-art KD baselines on one-stage, two-stage and anchor-free object detectors.
  arXiv  Detail & Related papers  (2022-09-20T16:36:28Z)
• A Unified Speaker Adaptation Approach for ASR [37.76683818356052]
  We propose a unified speaker adaptation approach consisting of feature adaptation and model adaptation. For feature adaptation, we employ a speaker-aware persistent memory model which generalizes better to unseen test speakers. For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture.
  arXiv  Detail & Related papers  (2021-10-16T10:48:52Z)
• Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification [66.62686601948455]
  We exploit the scope of the transformer distillation method that is specifically designed for knowledge distillation from a transformer based language model to a transformer based speech model. We achieve an intent classification accuracy of 99.10% and 88.79% for Fluent speech corpus and ATIS database, respectively.
  arXiv  Detail & Related papers  (2021-08-05T13:08:13Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)
• Bayesian Learning for Deep Neural Network Adaptation [57.70991105736059]
  A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. This paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty.
  arXiv  Detail & Related papers  (2020-12-14T12:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.