An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory

• URL: http://arxiv.org/abs/2607.11707v2
• Date: Wed, 15 Jul 2026 14:17:22 GMT
• Title: An Explainable Agentic System for Detection of Conversational Scams with Summary-Based Memory
• Abstract summary: This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams.<n>ConScamBench-278 is an initial public multi-category benchmark for conversational scam detection spanning eight scam types.<n>On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus.
• Score: 0.7712979281505654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Following the rapid progress of generative Artificial Intelligence, there is a growing threat posed by conversational scams. These scams often span over multiple weeks or months, gradually build trust and request for money or sensitive information. Existing scam-detection systems mainly focus on isolated messages, which renders them inadequate against this evolving threat. This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams. It also introduces ConScamBench-278, an initial public multi-category benchmark for conversational scam detection spanning eight scam types, released to support reproducible evaluation and future expansion. On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus (83/83) and reaches 97.8% accuracy (95% CI [95.4, 99.0]) on ConScamBench-278. Two user studies (N = 100 and N = 45) further motivate the system: participants report frequently experiencing uncertainty when judging suspicious conversations. In an uncontrolled pre/post comparison, users self-reported trust, self-confidence, and perceived need for AI-based scam detection all increased (p < 0.001, Wilcoxon signed-rank). The system also receives a System Usability Scale score of 74.7 (95% CI [72.5, 76.9]), above the established usability benchmark.

Related papers

• Black-Box Forensics for Conversational LLM Agents [27.24002262427959]
  Black-box forensics for conversational LLM agents offers a path to accountability for systems hidden behind anonymous endpoints.<n>We identify the base model behind an agent with 98% accuracy from a few turns of non-adversarial conversation.<n>Our cross-encoder fingerprinting method achieves an AUC of 0.768 and an F1 of 0.703 on entirely unseen system prompts.
  arXiv  Detail & Related papers  (2026-06-21T22:24:55Z)
• PreScam: A Benchmark for Predicting Scam Progression from Early Conversations [22.864427809364397]
  We introduce PreScam, a benchmark for modeling scam progression from early conversations.<n>We benchmark models on two tasks: real-time termination prediction and scammer action prediction.<n>Results show a clear gap between surface-level fluency and progression modeling.
  arXiv  Detail & Related papers  (2026-05-12T15:11:41Z)
• How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition [48.32744727426218]
  LLM based agents are increasingly deployed in high stakes settings where they process external data sources such as emails, documents, and code repositories.<n>This creates exposure to indirect prompt injection attacks, where adversarial instructions embedded in external content manipulate agent behavior without user awareness.<n>We present findings from a large scale public red teaming competition evaluating this dual objective across three agent settings.
  arXiv  Detail & Related papers  (2026-03-16T14:49:36Z)
• SAFE-QAQ: End-to-End Slow-Thinking Audio-Text Fraud Detection via Reinforcement Learning [52.29460857893198]
  Existing fraud detection methods rely on transcribed text, suffering from ASR errors and missing crucial acoustic cues like vocal tone and environmental context.<n>We propose SAFE-QAQ, an end-to-end comprehensive framework for audio-based slow-thinking fraud detection.<n>Our framework introduces a dynamic risk assessment framework during live calls, enabling early detection and prevention of fraud.
  arXiv  Detail & Related papers  (2026-01-04T06:09:07Z)
• Send to which account? Evaluation of an LLM-based Scambaiting System [0.0]
  This paper presents the first large-scale, real-world evaluation of a scambaiting system powered by large language models (LLMs)<n>Over a five-month deployment, the system initiated over 2,600 engagements with actual scammers, resulting in a dataset of more than 18,700 messages.<n>It achieved an Information Disclosure Rate (IDR) of approximately 32%, successfully extracting sensitive financial information such as mule accounts.
  arXiv  Detail & Related papers  (2025-09-10T11:08:52Z)
• CASE: An Agentic AI Framework for Enhancing Scam Intelligence in Digital Payments [0.24378845585726894]
  This paper presents CASE (Conversational Agent for Scam Elucidation), a novel Agentic AI framework.<n>A conversational agent is uniquely designed to proactively interview potential victims to elicit intelligence in the form of a detailed conversation.<n>By augmenting our existing features with this new intelligence, we have observed a 21% uplift in the volume of scam enforcements.
  arXiv  Detail & Related papers  (2025-08-27T14:47:33Z)
• Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions [89.35345649303451]
  Generative search engines have the potential to transform how people seek information online. But generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate. Retrieval-augmented generation exacerbates safety concerns, since adversaries may successfully evade the entire system.
  arXiv  Detail & Related papers  (2024-02-25T11:22:19Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Automatic Scam-Baiting Using ChatGPT [0.46040036610482665]
  We report on the results of a month-long experiment comparing the effectiveness of two ChatGPT-based automatic scam-baiters to a control measure. With engagement from over 250 real email fraudsters, we find that ChatGPT-based scam-baiters show a marked increase in scammer response rate and conversation length. We discuss the implications of these results and practical considerations for wider deployment of automatic scam-baiting.
  arXiv  Detail & Related papers  (2023-09-04T13:13:35Z)
• FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases [50.065022493142116]
  Trojan attack on deep neural networks, also known as backdoor attack, is a typical threat to artificial intelligence. FreeEagle is the first data-free backdoor detection method that can effectively detect complex backdoor attacks.
  arXiv  Detail & Related papers  (2023-02-28T11:31:29Z)
• DFraud3- Multi-Component Fraud Detection freeof Cold-start [50.779498955162644]
  The Cold-start is a significant problem referring to the failure of a detection system to recognize the authenticity of a new user. In this paper, we model a review system as a Heterogeneous InformationNetwork (HIN) which enables a unique representation to every component. HIN with graph induction helps to address the camouflage issue (fraudsterswith genuine reviews) which has shown to be more severe when it is coupled with cold-start, i.e., new fraudsters with genuine first reviews.
  arXiv  Detail & Related papers  (2020-06-10T08:20:13Z)
• Integrated Replay Spoofing-aware Text-independent Speaker Verification [47.41124427552161]
  We propose two approaches for building an integrated system of speaker verification and presentation attack detection. The first approach simultaneously trains speaker identification, presentation attack detection, and the integrated system using multi-task learning. We propose a back-end modular approach using a separate deep neural network (DNN) for speaker verification and presentation attack detection.
  arXiv  Detail & Related papers  (2020-06-10T01:24:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.