SPBA: Utilizing Speech Large Language Model for Backdoor Attacks on Speech Classification Models

• URL: http://arxiv.org/abs/2506.08346v1
• Date: Tue, 10 Jun 2025 02:01:00 GMT
• Title: SPBA: Utilizing Speech Large Language Model for Backdoor Attacks on Speech Classification Models
• Authors: Wenhan Yao, Fen Xiao, Xiarun Chen, Jia Liu, YongQiang He, Weiping Wen,
• Abstract summary: Speech-based human-computer interaction is vulnerable to backdoor attacks.<n>In this paper, we propose that speech backdoor attacks can strategically focus on speech elements such as timbre and emotion.<n>The proposed attack is called the Speech Prompt Backdoor Attack (SPBA)
• Score: 4.67675814519416
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Deep speech classification tasks, including keyword spotting and speaker verification, are vital in speech-based human-computer interaction. Recently, the security of these technologies has been revealed to be susceptible to backdoor attacks. Specifically, attackers use noisy disruption triggers and speech element triggers to produce poisoned speech samples that train models to become vulnerable. However, these methods typically create only a limited number of backdoors due to the inherent constraints of the trigger function. In this paper, we propose that speech backdoor attacks can strategically focus on speech elements such as timbre and emotion, leveraging the Speech Large Language Model (SLLM) to generate diverse triggers. Increasing the number of triggers may disproportionately elevate the poisoning rate, resulting in higher attack costs and a lower success rate per trigger. We introduce the Multiple Gradient Descent Algorithm (MGDA) as a mitigation strategy to address this challenge. The proposed attack is called the Speech Prompt Backdoor Attack (SPBA). Building on this foundation, we conducted attack experiments on two speech classification tasks, demonstrating that SPBA shows significant trigger effectiveness and achieves exceptional performance in attack metrics.

Related papers

• Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning [5.0734761482919115]
  Open-vocabulary object detectors (OVODs) unify vision and language to detect arbitrary object categories based on text prompts.<n>We conduct the first study of backdoor attacks on OVODs and reveal a new attack surface introduced by prompt tuning.
  arXiv  Detail & Related papers  (2025-11-16T19:05:31Z)
• MTAttack: Multi-Target Backdoor Attacks against Large Vision-Language Models [52.37749859972453]
  We propose MTAttack, the first multi-target backdoor attack framework for enforcing accurate multiple trigger-target mappings in LVLMs.<n> Experiments on popular benchmarks demonstrate a high success rate of MTAttack for multi-target attacks.<n>Our attack exhibits strong generalizability across datasets and robustness against backdoor defense strategies.
  arXiv  Detail & Related papers  (2025-11-13T09:00:21Z)
• Backdoor Attacks Against Speech Language Models [63.07317091368079]
  We present the first systematic study of audio backdoor attacks against speech language models.<n>We demonstrate its effectiveness across four speech encoders and three datasets, covering four tasks.<n>We propose a fine-tuning-based defense that mitigates the threat of poisoned pretrained encoders.
  arXiv  Detail & Related papers  (2025-10-01T17:45:04Z)
• Wolf Hidden in Sheep's Conversations: Toward Harmless Data-Based Backdoor Attacks for Jailbreaking Large Language Models [69.11679786018206]
  Supervised fine-tuning (SFT) aligns large language models with human intent by training them on labeled task-specific data.<n>Recent studies have shown that malicious attackers can inject backdoors into these models by embedding triggers into the harmful question-answer pairs.<n>We propose a novel clean-data backdoor attack for jailbreaking LLMs.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs [6.8285467057172555]
  We investigate universal acoustic adversarial attacks on speech LLMs.<n>We find critical vulnerabilities in Qwen2-Audio and Granite-Speech.<n>This highlights the need for more robust training strategies and improved resistance to adversarial attacks.
  arXiv  Detail & Related papers  (2025-05-20T12:35:59Z)
• EmoAttack: Utilizing Emotional Voice Conversion for Speech Backdoor Attacks on Deep Speech Classification Models [4.164975438207411]
  Speech backdoor attacks can strategically focus on emotion, a higher-level subjective perceptual attribute inherent in speech. EmoAttack method owns impactful trigger effectiveness and its remarkable attack success rate and accuracy variance.
  arXiv  Detail & Related papers  (2024-08-28T03:36:43Z)
• LOTUS: Evasive and Resilient Backdoor Attacks through Sub-Partitioning [49.174341192722615]
  Backdoor attack poses a significant security threat to Deep Learning applications. Recent papers have introduced attacks using sample-specific invisible triggers crafted through special transformation functions. We introduce a novel backdoor attack LOTUS to address both evasiveness and resilience.
  arXiv  Detail & Related papers  (2024-03-25T21:01:29Z)
• Does Few-shot Learning Suffer from Backdoor Attacks? [63.9864247424967]
  We show that few-shot learning can still be vulnerable to backdoor attacks. Our method demonstrates a high Attack Success Rate (ASR) in FSL tasks with different few-shot learning paradigms. This study reveals that few-shot learning still suffers from backdoor attacks, and its security should be given attention.
  arXiv  Detail & Related papers  (2023-12-31T06:43:36Z)
• Attention-Enhancing Backdoor Attacks Against BERT-based Models [54.070555070629105]
  Investigating the strategies of backdoor attacks will help to understand the model's vulnerability. We propose a novel Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating the attention patterns.
  arXiv  Detail & Related papers  (2023-10-23T01:24:56Z)
• Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound [9.24846124692153]
  Deep neural networks (DNNs) have been widely and successfully adopted and deployed in various applications of speech recognition. In this paper, we revisit poison-only backdoor attacks against speech recognition. We exploit elements of sound ($e.g.$, pitch and timbre) to design more stealthy yet effective poison-only backdoor attacks.
  arXiv  Detail & Related papers  (2023-07-17T02:58:25Z)
• Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion [14.264424889358208]
  This work explores a backdoor attack that utilizes sample-specific triggers based on voice conversion. Specifically, we adopt a pre-trained voice conversion model to generate the trigger, ensuring that the poisoned samples does not introduce any additional audible noise.
  arXiv  Detail & Related papers  (2023-06-28T02:19:31Z)
• From Shortcuts to Triggers: Backdoor Defense with Denoised PoE [51.287157951953226]
  Language models are often at risk of diverse backdoor attacks, especially data poisoning. Existing backdoor defense methods mainly focus on backdoor attacks with explicit triggers. We propose an end-to-end ensemble-based backdoor defense framework, DPoE, to defend various backdoor attacks.
  arXiv  Detail & Related papers  (2023-05-24T08:59:25Z)
• Backdoor Attacks with Input-unique Triggers in NLP [34.98477726215485]
  Backdoor attack aims at inducing neural models to make incorrect predictions for poison data while keeping predictions on the clean dataset unchanged. In this paper, we propose an input-unique backdoor attack(NURA), where we generate backdoor triggers unique to inputs.
  arXiv  Detail & Related papers  (2023-03-25T01:41:54Z)
• Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger [48.59965356276387]
  We propose to use syntactic structure as the trigger in textual backdoor attacks. We conduct extensive experiments to demonstrate that the trigger-based attack method can achieve comparable attack performance. These results also reveal the significant insidiousness and harmfulness of textual backdoor attacks.
  arXiv  Detail & Related papers  (2021-05-26T08:54:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.