ASRJam: Human-Friendly AI Speech Jamming to Prevent Automated Phone Scams

• URL: http://arxiv.org/abs/2506.11125v1
• Date: Tue, 10 Jun 2025 10:04:23 GMT
• Title: ASRJam: Human-Friendly AI Speech Jamming to Prevent Automated Phone Scams
• Authors: Freddie Grabovski, Gilad Gressel, Yisroel Mirsky,
• Abstract summary: Large Language Models (LLMs), combined with Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), are increasingly used to automate voice phishing (vishing) scams.<n>We introduce ASRJam, a proactive defence framework that injects adversarial perturbations into the victim's audio to disrupt the attacker's ASR.<n>We also propose EchoGuard, a novel jammer that leverages natural distortions, such as reverberation and echo, that are disruptive to ASR but tolerable to humans.
• Score: 2.6528263069045126
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Models (LLMs), combined with Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), are increasingly used to automate voice phishing (vishing) scams. These systems are scalable and convincing, posing a significant security threat. We identify the ASR transcription step as the most vulnerable link in the scam pipeline and introduce ASRJam, a proactive defence framework that injects adversarial perturbations into the victim's audio to disrupt the attacker's ASR. This breaks the scam's feedback loop without affecting human callers, who can still understand the conversation. While prior adversarial audio techniques are often unpleasant and impractical for real-time use, we also propose EchoGuard, a novel jammer that leverages natural distortions, such as reverberation and echo, that are disruptive to ASR but tolerable to humans. To evaluate EchoGuard's effectiveness and usability, we conducted a 39-person user study comparing it with three state-of-the-art attacks. Results show that EchoGuard achieved the highest overall utility, offering the best combination of ASR disruption and human listening experience.

Related papers

• Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World [54.68651652564436]
  This paper investigates the real-world vulnerabilities of audio-based large language models (ALLMs), such as Qwen2-Audio.<n>An adversary can craft stealthy audio perturbations to manipulate ALLMs into exhibiting specific targeted behaviors.<n>We show that playing adversarial background noise during user interaction with the ALLMs can significantly degrade the response quality.
  arXiv  Detail & Related papers  (2025-07-07T07:29:52Z)
• SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis [8.590034271906289]
  Speech synthesis technology has brought great convenience, while the widespread usage of realistic deepfake audio has triggered hazards.<n>Malicious adversaries may unauthorizedly collect victims' speeches and clone a similar voice for illegal exploitation.<n>We propose a framework, textittextbfSafeSpeech, which protects the users' audio before uploading by embedding imperceptible perturbations on original speeches.
  arXiv  Detail & Related papers  (2025-04-14T03:21:23Z)
• Whispering Under the Eaves: Protecting User Privacy Against Commercial and LLM-powered Automatic Speech Recognition Systems [20.45938874279563]
  We propose a novel framework, AudioShield, to protect speech against automatic speech recognition systems.<n>By transferring the perturbations to the latent space, the audio quality is preserved to a large extent.<n> AudioShield shows high effectiveness in real-time end-to-end scenarios, and demonstrates strong resilience against adaptive countermeasures.
  arXiv  Detail & Related papers  (2025-04-01T14:49:39Z)
• Predictive Speech Recognition and End-of-Utterance Detection Towards Spoken Dialog Systems [55.99999020778169]
  We study a function that can predict the forthcoming words and estimate the time remaining until the end of an utterance. We develop a cross-attention-based algorithm that incorporates both acoustic and linguistic information. Results demonstrate the proposed model's ability to predict upcoming words and estimate future EOU events up to 300ms prior to the actual EOU.
  arXiv  Detail & Related papers  (2024-09-30T06:29:58Z)
• Towards Unsupervised Speech Recognition Without Pronunciation Models [57.222729245842054]
  In this article, we tackle the challenge of developing ASR systems without paired speech and text corpora.<n>We experimentally demonstrate that an unsupervised speech recognizer can emerge from joint speech-to-speech and text-to-text masked token-infilling.<n>This innovative model surpasses the performance of previous unsupervised ASR models under the lexicon-free setting.
  arXiv  Detail & Related papers  (2024-06-12T16:30:58Z)
• Towards Evaluating the Robustness of Automatic Speech Recognition Systems via Audio Style Transfer [8.948537516293328]
  We propose an attack on Automatic Speech Recognition (ASR) systems based on user-customized style transfer. Our method can meet the need for user-customized styles and achieve a success rate of 82% in attacks.
  arXiv  Detail & Related papers  (2024-05-15T16:05:24Z)
• Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection [88.74863771919445]
  We reveal the vulnerability of AVASD models under audio-only, visual-only, and audio-visual adversarial attacks. We also propose a novel audio-visual interaction loss (AVIL) for making attackers difficult to find feasible adversarial examples.
  arXiv  Detail & Related papers  (2022-10-03T08:10:12Z)
• Blackbox Untargeted Adversarial Testing of Automatic Speech Recognition Systems [1.599072005190786]
  Speech recognition systems are prevalent in applications for voice navigation and voice control of domestic appliances. Deep neural networks (DNNs) have been shown to be susceptible to adversarial perturbations. To help test the correctness of ASRS, we propose techniques that automatically generate blackbox.
  arXiv  Detail & Related papers  (2021-12-03T10:21:47Z)
• WaveGuard: Understanding and Mitigating Audio Adversarial Examples [12.010555227327743]
  We introduce WaveGuard: a framework for detecting adversarial inputs crafted to attack ASR systems. Our framework incorporates audio transformation functions and analyses the ASR transcriptions of the original and transformed audio to detect adversarial inputs.
  arXiv  Detail & Related papers  (2021-03-04T21:44:37Z)
• Dompteur: Taming Audio Adversarial Examples [28.54699912239861]
  Adversarial examples allow attackers to arbitrarily manipulate machine learning systems. In this paper we propose a different perspective: We accept the presence of adversarial examples against ASR systems, but we require them to be perceivable by human listeners. By applying the principles of psychoacoustics, we can remove semantically irrelevant information from the ASR input and train a model that resembles human perception more closely.
  arXiv  Detail & Related papers  (2021-02-10T13:53:32Z)
• Cortical Features for Defense Against Adversarial Audio Attacks [55.61885805423492]
  We propose using a computational model of the auditory cortex as a defense against adversarial attacks on audio. We show that the cortical features help defend against universal adversarial examples.
  arXiv  Detail & Related papers  (2021-01-30T21:21:46Z)
• Speaker De-identification System using Autoencoders and Adversarial Training [58.720142291102135]
  We propose a speaker de-identification system based on adversarial training and autoencoders. Experimental results show that combining adversarial learning and autoencoders increase the equal error rate of a speaker verification system.
  arXiv  Detail & Related papers  (2020-11-09T19:22:05Z)
• Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement [102.48582597586233]
  We present a U-Net based attention model, U-Net$_At$, to enhance adversarial speech signals. We conduct experiments on the automatic speech recognition (ASR) task with adversarial audio attacks.
  arXiv  Detail & Related papers  (2020-03-31T02:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.