PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting

• URL: http://arxiv.org/abs/2603.18023v1
• Date: Thu, 05 Mar 2026 13:57:52 GMT
• Title: PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting
• Abstract summary: This paper introduces a multi-task learning framework for personalized, customizable open-vocabulary Keyword Spotting (PCOV-KWS)<n>We have integrated a training criterion distinct from softmax-based loss, transforming multi-class classification into multiple binary classifications, which eliminates inter-category competition.<n>We evaluated our PCOV-KWS system in multiple datasets, demonstrating that it outperforms the baselines in evaluation results.
• Score: 7.950602766668019
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As advancements in technologies like Internet of Things (IoT), Automatic Speech Recognition (ASR), Speaker Verification (SV), and Text-to-Speech (TTS) lead to increased usage of intelligent voice assistants, the demand for privacy and personalization has escalated. In this paper, we introduce a multi-task learning framework for personalized, customizable open-vocabulary Keyword Spotting (PCOV-KWS). This framework employs a lightweight network to simultaneously perform Keyword Spotting (KWS) and SV to address personalized KWS requirements. We have integrated a training criterion distinct from softmax-based loss, transforming multi-class classification into multiple binary classifications, which eliminates inter-category competition, while an optimization strategy for multi-task loss weighting is employed during training. We evaluated our PCOV-KWS system in multiple datasets, demonstrating that it outperforms the baselines in evaluation results, while also requiring fewer parameters and lower computational resources.

Related papers

• Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection [89.42023974249122]
  Adapt-$infty$ is a new multi-way and adaptive data selection approach for lifelong instruction tuning.<n>We construct pseudo-skill clusters by grouping gradient-based sample vectors.<n>We select the best-performing data selector for each skill cluster from a pool of selector experts.<n>This data selector samples a subset of the most important samples from each skill cluster for training.
  arXiv  Detail & Related papers  (2024-10-14T15:48:09Z)
• Semantic Meta-Split Learning: A TinyML Scheme for Few-Shot Wireless Image Classification [50.28867343337997]
  This work presents a TinyML-based semantic communication framework for few-shot wireless image classification. We exploit split-learning to limit the computations performed by the end-users while ensuring privacy-preserving. meta-learning overcomes data availability concerns and speeds up training by utilizing similarly trained tasks.
  arXiv  Detail & Related papers  (2024-09-03T05:56:55Z)
• Efficient Spoken Language Recognition via Multilabel Classification [53.662747523872305]
  We show that our models obtain competitive results while being orders of magnitude smaller and faster than current state-of-the-art methods. Our multilabel strategy is more robust to unseen non-target languages compared to multiclass classification.
  arXiv  Detail & Related papers  (2023-06-02T23:04:19Z)
• Personalized Keyword Spotting through Multi-task Learning [6.4423565043274795]
  We design two personalized KWS tasks; (1) Target user Biased KWS (TB-KWS) and (2) Target user Only KWS (TO-KWS) To solve the tasks, we propose personalized keyword spotting through multi-task learning (PK-MTL) that consists of multi-task learning and task-adaptation. We evaluate our framework on conventional and personalized scenarios, and the results show that PK-MTL can dramatically reduce the false alarm rate.
  arXiv  Detail & Related papers  (2022-06-28T02:48:34Z)
• Learning Decoupling Features Through Orthogonality Regularization [55.79910376189138]
  Keywords spotting (KWS) and speaker verification (SV) are two important tasks in speech applications. We develop a two-branch deep network (KWS branch and SV branch) with the same network structure. A novel decoupling feature learning method is proposed to push up the performance of KWS and SV simultaneously.
  arXiv  Detail & Related papers  (2022-03-31T03:18:13Z)
• Generalizing AUC Optimization to Multiclass Classification for Audio Segmentation With Limited Training Data [20.943224434364517]
  Area under the ROC curve (AUC) optimisation techniques have recently demonstrated their capabilities in different audio and speech related tasks. We introduce an extension to the AUC optimisation framework so that it can be easily applied to an arbitrary number of classes.
  arXiv  Detail & Related papers  (2021-10-27T13:36:04Z)
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing [102.45426364965887]
  We propose a new pre-trained model, WavLM, to solve full-stack downstream speech tasks. WavLM is built based on the HuBERT framework, with an emphasis on both spoken content modeling and speaker identity preservation. We scale up the training dataset from 60k hours to 94k hours of public audio data, and optimize its training procedure for better representation extraction.
  arXiv  Detail & Related papers  (2021-10-26T17:55:19Z)
• Multi-task Learning with Cross Attention for Keyword Spotting [8.103605110339519]
  Keywords spotting (KWS) is an important technique for speech applications, which enables users to activate devices by speaking a keyword phrase. There is a mismatch between the training criterion (phoneme recognition) and the target task (KWS) Recently, multi-task learning has been applied to KWS to exploit both ASR and KWS training data.
  arXiv  Detail & Related papers  (2021-07-15T22:38:16Z)
• Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition [62.94773371761236]
  We consider building an effective end-to-end ASR system in low-resource setups with a high OOV rate. We propose a method of dynamic acoustic unit augmentation based on the BPE-dropout technique. Our monolingual Turkish Conformer established a competitive result with 22.2% character error rate (CER) and 38.9% word error rate (WER)
  arXiv  Detail & Related papers  (2021-03-12T10:10:13Z)
• Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks [3.8382752162527933]
  In this paper, we focus on an open-vocabulary keyword spotting method, allowing the user to define their own keywords without having to retrain the whole model. We describe the different design choices leading to a fast and small-footprint system, able to run on tiny devices, for any arbitrary set of user-defined keywords.
  arXiv  Detail & Related papers  (2020-02-25T13:27:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.