Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams

• URL: http://arxiv.org/abs/2503.07036v1
• Date: Mon, 10 Mar 2025 08:21:36 GMT
• Title: Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams
• Authors: Nardine Basta, Conor Atkins, Dali Kaafar,
• Abstract summary: "Bot Wars" is a framework using Large Language Models (LLMs) scam-baiters to counter phone scams through simulated adversarial dialogues.<n>We evaluate our approach using a dataset of 3,200 scam dialogues validated against 179 hours of human scam-baiting interactions.
• Score: 0.8466004732265869
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present "Bot Wars," a framework using Large Language Models (LLMs) scam-baiters to counter phone scams through simulated adversarial dialogues. Our key contribution is a formal foundation for strategy emergence through chain-of-thought reasoning without explicit optimization. Through a novel two-layer prompt architecture, our framework enables LLMs to craft demographically authentic victim personas while maintaining strategic coherence. We evaluate our approach using a dataset of 3,200 scam dialogues validated against 179 hours of human scam-baiting interactions, demonstrating its effectiveness in capturing complex adversarial dynamics. Our systematic evaluation through cognitive, quantitative, and content-specific metrics shows that GPT-4 excels in dialogue naturalness and persona authenticity, while Deepseek demonstrates superior engagement sustainability.

Related papers

• SCRIPTMIND: Crime Script Inference and Cognitive Evaluation for LLM-based Social Engineering Scam Detection System [14.704193787708872]
  Social engineering scams increasingly employ personalized, multi-turn deception.<n>While Large Language Models (LLMs) show promise in identifying deception, their cognitive assistance potential remains underexplored.<n>We propose ScriptMind, an integrated framework for LLM-based scam detection that bridges automated reasoning and human cognition.
  arXiv  Detail & Related papers  (2026-01-20T04:11:00Z)
• The Imitation Game: Using Large Language Models as Chatbots to Combat Chat-Based Cybercrimes [24.05325129572158]
  Chat-based cybercrime has emerged as a pervasive threat.<n>Traditional defense mechanisms struggle to identify these conversational threats.<n>We present LURE, the first system to deploy Large Language Models as active agents.
  arXiv  Detail & Related papers  (2025-12-24T05:34:05Z)
• AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing [79.0112532518727]
  We release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain.<n>We then propose AI-Salesman, a novel framework featuring a dual-stage architecture.<n>We show that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations.
  arXiv  Detail & Related papers  (2025-11-15T09:44:42Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• ASRJam: Human-Friendly AI Speech Jamming to Prevent Automated Phone Scams [2.6528263069045126]
  Large Language Models (LLMs), combined with Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), are increasingly used to automate voice phishing (vishing) scams.<n>We introduce ASRJam, a proactive defence framework that injects adversarial perturbations into the victim's audio to disrupt the attacker's ASR.<n>We also propose EchoGuard, a novel jammer that leverages natural distortions, such as reverberation and echo, that are disruptive to ASR but tolerable to humans.
  arXiv  Detail & Related papers  (2025-06-10T10:04:23Z)
• SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues [9.762621950740995]
  Malicious attackers can exploit large language models (LLMs) by engaging them in multi-turn dialogues.<n>We propose a novel defense mechanism: SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues (STREAM)
  arXiv  Detail & Related papers  (2025-05-31T18:38:23Z)
• On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks [8.28564202645918]
  We propose a framework for orchestrating AR-driven Social Engineering attacks using Multimodal Large Language Models. Our results show that SEAR is highly effective at eliciting high-risk behaviors. We identify notable limitations such as occasionally artificial'' due to perceived authenticity gaps.
  arXiv  Detail & Related papers  (2025-04-16T05:18:36Z)
• Personalized Attacks of Social Engineering in Multi-turn Conversations -- LLM Agents for Simulation and Detection [19.625518218365382]
  Social engineering (SE) attacks on social media platforms pose a significant risk. We propose an LLM-agentic framework, SE-VSim, to simulate SE attack mechanisms by generating multi-turn conversations. We present a proof of concept, SE-OmniGuard, to offer personalized protection to users by leveraging prior knowledge of the victims personality.
  arXiv  Detail & Related papers  (2025-03-18T19:14:44Z)
• REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation [51.97224538045096]
  We introduce REALTALK, a 21-day corpus of authentic messaging app dialogues.<n>We compare EI attributes and persona consistency to understand the challenges posed by real-world dialogues.<n>Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation.
  arXiv  Detail & Related papers  (2025-02-18T20:29:01Z)
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework.<n>It reformulates harmful queries into benign reasoning tasks.<n>We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• Verbalized Bayesian Persuasion [54.55974023595722]
  Information design (ID) explores how a sender influence the optimal behavior of receivers to achieve specific objectives.<n>This work proposes a verbalized framework in Bayesian persuasion (BP), which extends classic BP to real-world games involving human dialogues for the first time.<n> Numerical experiments in dialogue scenarios, such as recommendation letters, courtroom interactions, and law enforcement, validate that our framework can both reproduce theoretical results in classic BP and discover effective persuasion strategies.
  arXiv  Detail & Related papers  (2025-02-03T18:20:10Z)
• Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context [49.13497493053742]
  Previous research on LLM vulnerabilities often relied on nonsensical adversarial prompts, which were easily detectable by automated methods.<n>We address this gap by focusing on human-readable adversarial prompts, a more realistic and potent threat.<n>Our key contributions are situation-driven attacks leveraging movie scripts to create contextually relevant, human-readable prompts that successfully deceive LLMs.
  arXiv  Detail & Related papers  (2024-12-20T21:43:52Z)
• Who Can Withstand Chat-Audio Attacks? An Evaluation Benchmark for Large Language Models [70.99768410765502]
  Adrial audio attacks pose a significant threat to the growing use of large language models (LLMs) in voice-based human-machine interactions. We introduce the Chat-Audio Attacks benchmark including four distinct types of audio attacks. We evaluate six state-of-the-art LLMs with voice interaction capabilities, including Gemini-1.5-Pro, GPT-4o, and others.
  arXiv  Detail & Related papers  (2024-11-22T10:30:48Z)
• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations [58.65755268815283]
  Many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.
  arXiv  Detail & Related papers  (2024-11-07T21:37:51Z)
• Defending Against Social Engineering Attacks in the Age of LLMs [19.364994678178036]
  Large Language Models (LLMs) can emulate human conversational patterns and facilitate chat-based social engineering (CSE) attacks. This study investigates the dual capabilities of LLMs as both facilitators and defenders against CSE threats. We propose ConvoSentinel, a modular defense pipeline that improves detection at both the message and the conversation levels.
  arXiv  Detail & Related papers  (2024-06-18T04:39:40Z)
• Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs [13.317364896194903]
  We propose a two-stage adversarial tuning framework to enhance Large Language Models' generalized defense capabilities. In the first stage, we introduce the hierarchical meta-universal adversarial prompt learning to efficiently generate token-level adversarial prompts. In the second stage, we propose the automatic adversarial prompt learning to iteratively refine semantic-level adversarial prompts.
  arXiv  Detail & Related papers  (2024-06-07T15:37:15Z)
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate [57.71597869337909]
  We build a multi-agent referee team called ChatEval to autonomously discuss and evaluate the quality of generated responses from different models. Our analysis shows that ChatEval transcends mere textual scoring, offering a human-mimicking evaluation process for reliable assessments.
  arXiv  Detail & Related papers  (2023-08-14T15:13:04Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.