A Practical Survey on Emerging Threats from AI-driven Voice Attacks: How Vulnerable are Commercial Voice Control Systems?

• URL: http://arxiv.org/abs/2312.06010v2
• Date: Thu, 4 Jan 2024 19:27:50 GMT
• Title: A Practical Survey on Emerging Threats from AI-driven Voice Attacks: How Vulnerable are Commercial Voice Control Systems?
• Authors: Yuanda Wang, Qiben Yan, Nikolay Ivanov, Xun Chen,
• Abstract summary: AI-driven audio attacks have revealed new security vulnerabilities in voice control systems. Our study endeavors to assess the resilience of commercial voice control systems against a spectrum of malicious audio attacks. Our results suggest that commercial voice control systems exhibit enhanced resistance to existing threats.
• Score: 13.115517847161428
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The emergence of Artificial Intelligence (AI)-driven audio attacks has revealed new security vulnerabilities in voice control systems. While researchers have introduced a multitude of attack strategies targeting voice control systems (VCS), the continual advancements of VCS have diminished the impact of many such attacks. Recognizing this dynamic landscape, our study endeavors to comprehensively assess the resilience of commercial voice control systems against a spectrum of malicious audio attacks. Through extensive experimentation, we evaluate six prominent attack techniques across a collection of voice control interfaces and devices. Contrary to prevailing narratives, our results suggest that commercial voice control systems exhibit enhanced resistance to existing threats. Particularly, our research highlights the ineffectiveness of white-box attacks in black-box scenarios. Furthermore, the adversaries encounter substantial obstacles in obtaining precise gradient estimations during query-based interactions with commercial systems, such as Apple Siri and Samsung Bixby. Meanwhile, we find that current defense strategies are not completely immune to advanced attacks. Our findings contribute valuable insights for enhancing defense mechanisms in VCS. Through this survey, we aim to raise awareness within the academic community about the security concerns of VCS and advocate for continued research in this crucial area.

Related papers

• Acoustic Cybersecurity: Exploiting Voice-Activated Systems [0.0]
  Our research extends the feasibility of these attacks across various platforms like Amazon's Alexa, Android, iOS, and Cortana. We quantitatively show that attack success rates hover around 60%, with the ability to activate devices remotely from over 100 feet away. These attacks threaten critical infrastructure, emphasizing the need for multifaceted defensive strategies.
  arXiv  Detail & Related papers  (2023-11-23T02:26:11Z)
• On the Security Risks of Knowledge Graph Reasoning [71.64027889145261]
  We systematize the security threats to KGR according to the adversary's objectives, knowledge, and attack vectors. We present ROAR, a new class of attacks that instantiate a variety of such threats. We explore potential countermeasures against ROAR, including filtering of potentially poisoning knowledge and training with adversarially augmented queries.
  arXiv  Detail & Related papers  (2023-05-03T18:47:42Z)
• Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection [88.74863771919445]
  We reveal the vulnerability of AVASD models under audio-only, visual-only, and audio-visual adversarial attacks. We also propose a novel audio-visual interaction loss (AVIL) for making attackers difficult to find feasible adversarial examples.
  arXiv  Detail & Related papers  (2022-10-03T08:10:12Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• SoK: A Study of the Security on Voice Processing Systems [2.596028864336544]
  We will identify and classify an arrangement of unique attacks on voice processing systems. The current and most frequently used machine learning systems and deep neural networks are at the core of modern voice processing systems.
  arXiv  Detail & Related papers  (2021-12-24T21:47:06Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Practical Attacks on Voice Spoofing Countermeasures [3.388509725285237]
  We show how a malicious actor may efficiently craft audio samples to bypass voice authentication in its strictest form. Our results call into question the security of modern voice authentication systems in light of the real threat of attackers bypassing these measures.
  arXiv  Detail & Related papers  (2021-07-30T14:07:49Z)
• Adversarial defense for automatic speaker verification by cascaded self-supervised learning models [101.42920161993455]
  More and more malicious attackers attempt to launch adversarial attacks at automatic speaker verification (ASV) systems. We propose a standard and attack-agnostic method based on cascaded self-supervised learning models to purify the adversarial perturbations. Experimental results demonstrate that the proposed method achieves effective defense performance and can successfully counter adversarial attacks.
  arXiv  Detail & Related papers  (2021-02-14T01:56:43Z)
• Adversarial Attack and Defense Strategies for Deep Speaker Recognition Systems [44.305353565981015]
  This paper considers several state-of-the-art adversarial attacks to a deep speaker recognition system, employing strong defense methods as countermeasures. Experiments show that the speaker recognition systems are vulnerable to adversarial attacks, and the strongest attacks can reduce the accuracy of the system from 94% to even 0%.
  arXiv  Detail & Related papers  (2020-08-18T00:58:19Z)
• Detecting Audio Attacks on ASR Systems with Dropout Uncertainty [40.9172128924305]
  We show that our defense is able to detect attacks created through optimized perturbations and frequency masking. We test our defense on Mozilla's CommonVoice dataset, the UrbanSound dataset, and an excerpt of the LibriSpeech dataset.
  arXiv  Detail & Related papers  (2020-06-02T19:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.