PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

• URL: http://arxiv.org/abs/2605.12243v1
• Date: Tue, 12 May 2026 15:11:41 GMT
• Title: PreScam: A Benchmark for Predicting Scam Progression from Early Conversations
• Abstract summary: We introduce PreScam, a benchmark for modeling scam progression from early conversations.<n>We benchmark models on two tasks: real-time termination prediction and scammer action prediction.<n>Results show a clear gap between surface-level fluency and progression modeling.
• Score: 22.864427809364397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, they unfold through multi-turn conversations in which scammers gradually manipulate victims using evolving psychological techniques. However, existing research mainly focuses on static scam detection or synthetic scams, leaving open whether language models can understand how real-world scams progress over time. We introduce PreScam, a benchmark for modeling scam progression from early conversations. Built from user-submitted scam reports, PreScam filters and structures 177,989 raw reports into 11,573 conversational scam instances spanning 20 scam categories. Each instance is hierarchically structured according to the scam lifecycle defined by the proposed scam kill chain, and further annotated at the turn level with scammer psychological actions and victim responses. We benchmark models on two tasks: real-time termination prediction, which estimates whether a conversation is approaching the termination stage, and scammer action prediction, which forecasts the scammer's subsequent actions. Results show a clear gap between surface-level fluency and progression modeling: supervised encoders substantially outperform zero-shot LLMs on real-time termination prediction, while next-action prediction remains only moderately successful even for strong LLMs. Taken together, these results show that current models can capture some scam-related cues, yet still struggle to track how risk escalates and how manipulation unfolds across turns.

Related papers

• DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots [49.59847654758852]
  Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs)<n>We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions.<n>We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning.
  arXiv  Detail & Related papers  (2026-08-05T16:11:08Z)
• Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings [85.35443905857876]
  We investigate turn-taking in multimodal multi-party conversations using large language models (LLMs)<n>We construct an evaluation framework for three tasks: addressee detection, turn-change prediction, and next speaker prediction.<n>We compare supervised models trained for these tasks, text-based LLMs, multimodal LLMs (MM-LLMs), and human subjects.
  arXiv  Detail & Related papers  (2026-06-16T05:46:44Z)
• CAMERA: Adapting to Semantic Camouflage in Unsupervised Text-Attributed Graph Fraud Detection [59.67216074776147]
  We propose a Case-Adaptive Multi-cue Expert fRAmework (CAMERA) for unsupervised TAGFD.<n>A context-informed gating model is introduced to jointly consider the ego node representation and its local neighborhood context.<n>Experiments on 4 challenging datasets show that CAMERA consistently outperforms competitors.
  arXiv  Detail & Related papers  (2026-05-19T15:54:09Z)
• ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usage [27.91201514467414]
  Smartphone scams are increasingly prevalent and typically manifest as multi-stage, cross-application processes.<n>We propose textbfORACLE Online Reasoning for Anticipating Cross-temporal LatentEats.
  arXiv  Detail & Related papers  (2026-05-09T16:26:16Z)
• SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimization [62.958457694151384]
  We introduce preference optimization into precipitation nowcasting for the first time, motivated by the success of reinforcement learning from human feedback in large language models.<n>In the first stage, the framework focuses on reducing FAR, training the model to effectively suppress false alarms.
  arXiv  Detail & Related papers  (2025-10-22T16:11:22Z)
• PsyScam: A Benchmark for Psychological Techniques in Real-World Scams [38.57446009573742]
  PsyScam is a benchmark designed to systematically capture the psychological techniques employed in real-world scam reports.<n>We show that PsyScam presents significant challenges to existing models in both detecting and generating scam content based on the PTs used by real-world scammers.
  arXiv  Detail & Related papers  (2025-05-21T01:55:04Z)
• "It Warned Me Just at the Right Moment": Exploring LLM-based Real-time Detection of Phone Scams [21.992539308179126]
  We propose a framework for modeling scam calls and introduce an LLM-based real-time detection approach.<n>We evaluate the method's performance and analyze key factors influencing its effectiveness.
  arXiv  Detail & Related papers  (2025-02-06T10:57:05Z)
• Exposing LLM Vulnerabilities: Adversarial Scam Detection and Performance [16.9071617169937]
  This paper investigates the vulnerabilities of Large Language Models (LLMs) when facing adversarial scam messages for the task of scam detection.<n>We created a comprehensive dataset with fine-grained labels of scam messages, including both original and adversarial scam messages.<n>Our analysis showed how adversarial examples took advantage of vulnerabilities of a LLM, leading to high misclassification rate.
  arXiv  Detail & Related papers  (2024-12-01T00:13:28Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)
• A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction [100.9772316028191]
  In this paper, we experiment with a variety of adversarial attack configurations to fool three stock prediction victim models. Our results show that the proposed attack method can achieve consistent success rates and cause significant monetary loss in trading simulation.
  arXiv  Detail & Related papers  (2022-05-01T05:12:22Z)
• Recent trends in Social Engineering Scams and Case study of Gift Card Scam [4.345672405192058]
  Social engineering scams (SES) has been existed since the adoption of the telecommunications by humankind. Recent trends of various social engineering scams targeting the innocent people all over the world. Case study of real-time gift card scam targeting various enterprise organization customers.
  arXiv  Detail & Related papers  (2021-10-13T04:17:02Z)
• Are socially-aware trajectory prediction models really socially-aware? [75.36961426916639]
  We introduce a socially-attended attack to assess the social understanding of prediction models. An attack is a small yet carefully-crafted perturbations to fail predictors. We show that our attack can be employed to increase the social understanding of state-of-the-art models.
  arXiv  Detail & Related papers  (2021-08-24T17:59:09Z)
• Adversarial Refinement Network for Human Motion Prediction [61.50462663314644]
  Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend. We propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation.
  arXiv  Detail & Related papers  (2020-11-23T05:42:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.