Training strategy for a lightweight countermeasure model for automatic speaker verification

• URL: http://arxiv.org/abs/2203.17031v1
• Date: Thu, 31 Mar 2022 13:52:43 GMT
• Title: Training strategy for a lightweight countermeasure model for automatic speaker verification
• Authors: Yen-Lun Liao, Xuanjun Chen, Chung-Che Wang, Jyh-Shing Roger Jang
• Abstract summary: This work proposes training strategies for a lightweight CM model for ASV. In the evalua- tion phase of the ASVspoof 2021 Logical Access task, the lightweight ResNetSE model reaches min t-DCF 0.2695 and EER 3.54%.
• Score: 6.174721516017139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The countermeasure (CM) model is developed to protect Automatic Speaker Verification (ASV) systems from spoof attacks and prevent resulting personal information leakage. Based on practicality and security considerations, the CM model is usually deployed on edge devices, which have more limited computing resources and storage space than cloud- based systems. This work proposes training strategies for a lightweight CM model for ASV, using generalized end- to-end (GE2E) pre-training and adversarial fine-tuning to improve performance, and applying knowledge distillation (KD) to reduce the size of the CM model. In the evalua- tion phase of the ASVspoof 2021 Logical Access task, the lightweight ResNetSE model reaches min t-DCF 0.2695 and EER 3.54%. Compared to the teacher model, the lightweight student model only uses 22.5% of parameters and 21.1% of multiply and accumulate operands of the teacher model.

Related papers

• Resource-Efficient Automatic Software Vulnerability Assessment via Knowledge Distillation and Particle Swarm Optimization [8.132644507041922]
  We propose a novel resource-efficient framework that integrates knowledge distillation and particle swarm optimization to enable automated vulnerability assessment.<n>Our framework employs a two-stage approach: First, particle swarm optimization is utilized to optimize the architecture of a compact student model.<n>Second, knowledge distillation is applied to transfer critical vulnerability assessment knowledge from a large teacher model to the optimized student model.
  arXiv  Detail & Related papers  (2025-07-30T13:55:28Z)
• Matryoshka Model Learning for Improved Elastic Student Models [62.154536258259384]
  MatTA is a framework for training multiple accurate Student models using a novel Teacher-TA-Student recipe.<n>We demonstrate our method on GPT-2 Medium, a public model, and achieve relative improvements of over 24% on SAT Math and over 10% on the LAMBADA benchmark.
  arXiv  Detail & Related papers  (2025-05-29T10:54:58Z)
• SHA256 at SemEval-2025 Task 4: Selective Amnesia -- Constrained Unlearning for Large Language Models via Knowledge Isolation [12.838593066237452]
  Large language models (LLMs) memorize frequently sensitive information during training, posing risks when deploying publicly accessible models. This paper presents our solution to SemEval-2025 Task 4 on targeted unlearning, which combines causal mediation analysis with layer-specific optimization.
  arXiv  Detail & Related papers  (2025-04-17T15:05:40Z)
• Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models [62.5501109475725]
  Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them. This paper introduces Online Knowledge Distillation (OKD), where the teacher network integrates small online modules to concurrently train with the student model. OKD achieves or exceeds the performance of leading methods in various model architectures and sizes, reducing training time by up to fourfold.
  arXiv  Detail & Related papers  (2024-09-19T07:05:26Z)
• Multi-Model based Federated Learning Against Model Poisoning Attack: A Deep Learning Based Model Selection for MEC Systems [11.564289367348334]
  Federated Learning (FL) enables training of a global model from distributed data, while preserving data privacy. This paper proposes a multi-model based FL as a proactive mechanism to enhance the opportunity of model poisoning attack mitigation. For a DDoS attack detection scenario, results illustrate a competitive accuracy gain under poisoning attack with the scenario that the system is without attack, and also a potential of recognition time improvement.
  arXiv  Detail & Related papers  (2024-09-12T17:36:26Z)
• CAMH: Advancing Model Hijacking Attack in Machine Learning [44.58778557522968]
  Category-Agnostic Model Hijacking (CAMH) is a novel model hijacking attack method. It addresses the challenges of class number mismatch, data distribution divergence, and performance balance between the original and hijacking tasks. We demonstrate its potent attack effectiveness while ensuring minimal degradation in the performance of the original task.
  arXiv  Detail & Related papers  (2024-08-25T07:03:01Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• Reusing Pretrained Models by Multi-linear Operators for Efficient Training [65.64075958382034]
  Training large models from scratch usually costs a substantial amount of resources. Recent studies such as bert2BERT and LiGO have reused small pretrained models to initialize a large model. We propose a method that linearly correlates each weight of the target model to all the weights of the pretrained model.
  arXiv  Detail & Related papers  (2023-10-16T06:16:47Z)
• Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models [103.71308117592963]
  We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning. In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification.
  arXiv  Detail & Related papers  (2022-11-27T21:43:45Z)
• p-Meta: Towards On-device Deep Model Adaptation [30.27192953408665]
  p-Meta is a new meta learning method that enforces structure-wise partial parameter updates while ensuring fast generalization to unseen tasks. We show that p-Meta substantially reduces the peak dynamic memory by a factor of 2.5 on average compared to state-of-the-art few-shot adaptation methods.
  arXiv  Detail & Related papers  (2022-06-25T18:36:59Z)
• Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic [67.00475077281212]
  Model-based reinforcement learning algorithms are more sample efficient than their model-free counterparts. We propose a novel approach that achieves high sample efficiency without the strong reliance on accurate learned models. We show that CMBAC significantly outperforms state-of-the-art approaches in terms of sample efficiency on several challenging tasks.
  arXiv  Detail & Related papers  (2021-12-16T15:33:11Z)
• Self-Feature Regularization: Self-Feature Distillation Without Teacher Models [0.0]
  Self-Feature Regularization(SFR) is proposed, which uses features in the deep layers to supervise feature learning in the shallow layers. We firstly use generalization-l2 loss to match local features and a many-to-one approach to distill more intensively in the channel dimension.
  arXiv  Detail & Related papers  (2021-03-12T15:29:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.