Evaluating AI Models' Capability to Automate Voice Phishing Attacks

• URL: http://arxiv.org/abs/2607.09970v1
• Date: Fri, 10 Jul 2026 20:54:32 GMT
• Title: Evaluating AI Models' Capability to Automate Voice Phishing Attacks
• Abstract summary: We conduct a large-scale survey experiment to assess U.S. adults' susceptibility to AI-powered voice phishing attacks.<n>Up to 36% of participants would or might comply with phishing requests in the "relative-in-distress" category.<n>Our economic analysis suggests that while human-operated vishing is unprofitable at US wages, AI-powered vishing appears to be economically viable for several models.
• Score: 5.7124370517630245
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Voice phishing (vishing) attacks have traditionally been limited by the need for human operators. The rapid emergence of high-quality AI voice synthesis and large language models (LLMs) reduces this bottleneck and enables scalable, automated scams. In this paper, we conduct a large-scale survey experiment (N=4100) and qualitative interviews (N=12) to assess U.S. adults' susceptibility to AI-powered voice phishing attacks. Participants were exposed to audio recordings or transcripts of scam scenarios generated using leading voice models such as Llama Full Duplex (Llama FD), Sesame, Gemini, OAI AVM, Play.AI, and ElevenLabs and the corresponding human baselines. The results show high compliance rates. Up to 36% of participants would or might comply with phishing requests in the "relative-in-distress" category. Overall compliance rate across all five scam categories was 16.5%, a striking figure given the low cost and high scalability of AI-automated voice phishing. Caller persuasiveness was the strongest predictor of compliance and certain models (most notably Sesame) achieved ratings comparable to human voices, or sometimes even slightly surpassing them. Our economic analysis suggests that while human-operated vishing is unprofitable at US wages, AI-powered vishing appears to be economically viable for several models. The primary risk of present-day AI-enabled vishing thus lies in the economics of automation rather than novel or "superhuman" persuasive techniques, though these cannot be ruled out for future systems. This raises significant concerns for the design of AI systems, consumer protection, and model release policies.

Related papers

• Characterizing initial human-AI proof formalization workflows [66.06866890889131]
  Advances in AI systems' ability to generate code and engage in increasingly high-level mathematical reasoning promise to transform people's ability to formalize and thereby verify proofs.<n>We conduct a mixed-methods analysis into the initial impact of AI on people's formalization.<n>A qualitative survey shows that people's preferences are diverse, but with a general desire for AI assistance in formalization.
  arXiv  Detail & Related papers  (2026-06-02T22:58:19Z)
• Can You Tell It's AI? Human Perception of Synthetic Voices in Vishing Scenarios [3.2976205772213123]
  Large Language Models and commercial speech synthesis systems now enable highly realistic AI-generated voice scams (vishing)<n>Yet it remains unclear whether individuals can reliably distinguish AI-generated speech from human-recorded voices in realistic scam contexts.<n>We conducted a controlled online study in which 22 participants evaluated 16 vishing-style audio clips and classified each as human or AI.
  arXiv  Detail & Related papers  (2026-02-23T17:17:53Z)
• Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians [47.64440749179653]
  We show that even an idealized Bayes-rational user is vulnerable to delusional spiraling.<n>This effect persists in the face of two candidate mitigations.
  arXiv  Detail & Related papers  (2026-02-22T12:13:44Z)
• Do AI Voices Learn Social Nuances? A Case of Politeness and Speech Rate [0.0]
  This study investigates whether state-of-the-art text-to-speech systems have the human tendency to reduce speech rate to convey politeness.<n>We prompted 22 synthetic voices from two leading AI platforms to read a fixed script under both "polite and formal" and "casual and informal" conditions.<n>Across both AI platforms, the polite prompt produced slower speech than the casual prompt with very large effect sizes.
  arXiv  Detail & Related papers  (2025-11-12T07:44:42Z)
• EQ-Negotiator: Dynamic Emotional Personas Empower Small Language Models for Edge-Deployable Credit Negotiation [66.09161596959771]
  Small language models (SLMs) offer a practical alternative, but suffer from a significant performance gap compared to large language models (LLMs)<n>This paper introduces EQ-Negotiator, a novel framework that bridges this capability gap using emotional personas.<n>We show that a 7B parameter language model with EQ-Negotiator achieves better debt recovery and negotiation efficiency than baseline LLMs more than 10 times its size.
  arXiv  Detail & Related papers  (2025-11-05T11:25:07Z)
• Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence [31.666988490509237]
  We show the pervasiveness and harmful impacts of sycophancy when people seek advice from AI.<n>We find that models are highly sycophantic, affirming users' actions 50% more than humans do.<n>Participants rated sycophantic responses as higher quality, trusted the sycophantic AI model more, and were more willing to use it again.
  arXiv  Detail & Related papers  (2025-10-01T19:26:01Z)
• ASRJam: Human-Friendly AI Speech Jamming to Prevent Automated Phone Scams [2.6528263069045126]
  Large Language Models (LLMs), combined with Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), are increasingly used to automate voice phishing (vishing) scams.<n>We introduce ASRJam, a proactive defence framework that injects adversarial perturbations into the victim's audio to disrupt the attacker's ASR.<n>We also propose EchoGuard, a novel jammer that leverages natural distortions, such as reverberation and echo, that are disruptive to ASR but tolerable to humans.
  arXiv  Detail & Related papers  (2025-06-10T10:04:23Z)
• Image Quality Assessment for Embodied AI [103.66095742463195]
  Embodied AI has developed rapidly in recent years, but it is still mainly deployed in laboratories.<n>There is no IQA method to assess the usability of an image in embodied tasks, namely, the perceptual quality for robots.
  arXiv  Detail & Related papers  (2025-05-22T15:51:07Z)
• AI persuading AI vs AI persuading Humans: LLMs' Differential Effectiveness in Promoting Pro-Environmental Behavior [70.24245082578167]
  Pro-environmental behavior (PEB) is vital to combat climate change, yet turning awareness into intention and action remains elusive.<n>We explore large language models (LLMs) as tools to promote PEB, comparing their impact across 3,200 participants.<n>Results reveal a "synthetic persuasion paradox": synthetic and simulated agents significantly affect their post-intervention PEB stance, while human responses barely shift.
  arXiv  Detail & Related papers  (2025-03-03T21:40:55Z)
• Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing [55.2480439325792]
  This study systematically evaluations twelve state-of-the-art AI-text detectors using our AI-Polished-Text Evaluation dataset.<n>Our findings reveal that detectors frequently flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models.
  arXiv  Detail & Related papers  (2025-02-21T18:45:37Z)
• AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test [0.0]
  GPT agents with varying levels of agreeableness were tested in a Turing Test. All exceeded a 50% confusion rate, with the highly agreeable AI agent surpassing 60%. This agent was also recognized as exhibiting the most human-like traits.
  arXiv  Detail & Related papers  (2024-11-20T23:12:49Z)
• PITCH: AI-assisted Tagging of Deepfake Audio Calls using Challenge-Response [14.604998731837595]
  We develop PITCH, a robust challenge-response method to detect and tag interactive deepfake audio calls.<n>PITCH's challenges enhanced machine detection capabilities to 88.7% AUROC score.<n>We develop a novel human-AI collaborative system that tags suspicious calls as "Deepfake-likely"
  arXiv  Detail & Related papers  (2024-02-28T06:17:55Z)
• Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap [56.611702960809644]
  We benchmark AI's ability to imitate humans in three language tasks and three vision tasks.<n>Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges.<n>Imitation ability showed minimal correlation with conventional AI performance metrics.
  arXiv  Detail & Related papers  (2022-11-23T16:16:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.