Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interaction

• URL: http://arxiv.org/abs/2608.10239v1
• Date: Mon, 10 Aug 2026 21:17:03 GMT
• Title: Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interaction
• Abstract summary: Generative AI makes social-engineering attacks more fluent, adaptive, and scalable.<n>We formalize trust-chain localization: identifying whether an interaction fails at actor authority, asset control, verification sufficiency, or transaction path.<n>We construct a controlled 300-case online-housing corpus spanning 20 scenario families, legitimate cases, four structural failure modes, and three surface conditions.
• Score: 42.06723421277899
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative AI makes social-engineering attacks more fluent, adaptive, and scalable, increasing the need for LLM-based de- fenders that can protect users during ongoing interactions. We ask whether such defenders identify the structural source of risk or merely react to surface cues. We formalize trust-chain localization: identifying whether an interaction fails at actor authority, asset control, verification sufficiency, or transaction path. We construct a controlled 300-case online-housing corpus spanning 20 scenario families, legitimate cases, four structural failure modes, and three surface conditions. Five defender models are evaluated on the same corpus in state- ful turn-by-turn and one-shot static settings, yielding 1,500 model-case evaluations per protocol and 3,000 in total. No model produced explicit unsafe compliance, yet defensive effectiveness varied sharply: intervention rates ranged from 0% to 96.3%. Protective action and correct structural localization were frequently decoupled, with models sometimes intervening while identifying the wrong trust component or recognizing a structural failure without taking protective action. Asset-control failures were a major localization bottleneck, surface sensitivity varied across models, and live-static differences were model-dependent. These findings show that safe-looking behavior alone is insufficient; live scam resistance must separately measure intervention, timing, structural localization, and false-positive behavior.

Related papers

• aiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-offs in LLM Safety, Security, and Privacy [6.323488051975665]
  AiXamine is a unified black-box platform that evaluates trustworthiness across safety, security, and privacy as interdependent properties.<n>AiXamine orchestrates 46 tests across nine services through an automated red-teaming pipeline.<n>We conduct the largest joint safety, security, and privacy study to date and uncover three cross-dimensional phenomena invisible to single-axis evaluation.
  arXiv  Detail & Related papers  (2026-08-20T20:33:35Z)
• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle [65.6642776933861]
  This survey examines the protective paradigm that has grown around this intervention point.<n>We find that most protections are still validated mainly against static or weakly adaptive adversaries.<n>We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
  arXiv  Detail & Related papers  (2026-08-05T00:46:50Z)
• Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels [0.0]
  This work proposes JADR, a protocol that measures a model's internal representation through Jacobian space.<n>For every prompt and layer we record the top-k J-space tokens.<n>These are grouped into six behavioral scenario axes and compared between a danger sample based on StrongREJECT and a safe control drawn from XSTest and OKTest.
  arXiv  Detail & Related papers  (2026-07-14T14:08:18Z)
• Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies [62.65225104423404]
  Self-evolving agent systems introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded.<n>We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface matrix.<n>Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent.
  arXiv  Detail & Related papers  (2026-06-22T09:23:50Z)
• The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents [0.0]
  Intervention timing is a low-reliability construct, making single-annotator F1 an unsuitable optimization target.<n>Our contribution is the joint mapping of this problem across human inter-rater reliability, four detector architectures, a cross-model LLM-judge sweep, and a reproduced saturation effect.
  arXiv  Detail & Related papers  (2026-06-02T23:54:27Z)
• Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs [20.59321114618083]
  We show that single-turn diagnostics systematically overestimate RAG safety, that contradiction is uncorrelated with safe resolution, and that no universal prompt fix exists.<n>The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.
  arXiv  Detail & Related papers  (2026-05-26T15:18:43Z)
• Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions [21.047807094226986]
  This paper presents a systematic survey of Large Language Models fine-tuning security.<n>It establishes a lifecycle-based framework for comparing attacks and defenses.<n>We identify key open problems (configuration-robust defense, cross-phase defense composition, and embedding-space attacks beyond behavioral assumptions) and propose concrete future research directions.
  arXiv  Detail & Related papers  (2026-05-24T13:34:47Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories [20.868825285848196]
  Safety guardrails are well-benchmarked for natural language responses, but their efficacy remains largely unexplored within multi-step tool-use trajectories.<n>To address this gap, we introduce StructuralSafe-Bench, the first comprehensive benchmark specifically designed to assess mid-trajectory safety.
  arXiv  Detail & Related papers  (2026-04-08T15:46:14Z)
• State-Dependent Safety Failures in Multi-Turn Language Model Interaction [70.52906620450847]
  We study safety failures from a state-space perspective and show that many multi-turn failures arise from structured contextual state evolution.<n>We introduce STAR, a state-oriented diagnostic framework that treats dialogue history as a state transition operator.<n>We find that systems that appear robust under static evaluation can undergo rapid and reproducible safety collapse under structured multi-turn interaction.
  arXiv  Detail & Related papers  (2026-03-15T12:13:01Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.