Debate-Driven Multi-Agent LLMs for Phishing Email Detection

• URL: http://arxiv.org/abs/2503.22038v1
• Date: Thu, 27 Mar 2025 23:18:14 GMT
• Title: Debate-Driven Multi-Agent LLMs for Phishing Email Detection
• Authors: Ngoc Tuong Vy Nguyen, Felix D Childress, Yunting Yin,
• Abstract summary: We propose a multi-agent large language model (LLM) prompting technique that simulates deceptive debates among agents to detect phishing emails.<n>Our approach uses two LLM agents to present arguments for or against the classification task, with a judge agent adjudicating the final verdict.<n>Results show that the debate structure itself is sufficient to yield accurate decisions without extra prompting strategies.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Phishing attacks remain a critical cybersecurity threat. Attackers constantly refine their methods, making phishing emails harder to detect. Traditional detection methods, including rule-based systems and supervised machine learning models, either rely on predefined patterns like blacklists, which can be bypassed with slight modifications, or require large datasets for training and still can generate false positives and false negatives. In this work, we propose a multi-agent large language model (LLM) prompting technique that simulates debates among agents to detect whether the content presented on an email is phishing. Our approach uses two LLM agents to present arguments for or against the classification task, with a judge agent adjudicating the final verdict based on the quality of reasoning provided. This debate mechanism enables the models to critically analyze contextual cue and deceptive patterns in text, which leads to improved classification accuracy. The proposed framework is evaluated on multiple phishing email datasets and demonstrate that mixed-agent configurations consistently outperform homogeneous configurations. Results also show that the debate structure itself is sufficient to yield accurate decisions without extra prompting strategies.

Related papers

• Enhancing Phishing Email Identification with Large Language Models [0.40792653193642503]
  We study the efficacy of large language models (LLMs) in detecting phishing emails. Experiments show that the LLM achieves a high accuracy rate at high precision.
  arXiv  Detail & Related papers  (2025-02-07T08:45:50Z)
• Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions [51.51850981481236]
  We introduce POATE, a novel jailbreak technique that harnesses contrastive reasoning to provoke unethical responses.<n>PoATE crafts semantically opposing intents and integrates them with adversarial templates, steering models toward harmful outputs with remarkable subtlety.<n>To counter this, we propose Intent-Aware CoT and Reverse Thinking CoT, which decompose queries to detect malicious intent and reason in reverse to evaluate and reject harmful responses.
  arXiv  Detail & Related papers  (2025-01-03T15:40:03Z)
• Next-Generation Phishing: How LLM Agents Empower Cyber Attackers [10.067883724547182]
  The escalating threat of phishing emails has become increasingly sophisticated with the rise of Large Language Models (LLMs) As attackers exploit LLMs to craft more convincing and evasive phishing emails, it is crucial to assess the resilience of current phishing defenses. We conduct a comprehensive evaluation of traditional phishing detectors, such as Gmail Spam Filter, Apache SpamAssassin, and Proofpoint, as well as machine learning models like SVM, Logistic Regression, and Naive Bayes. Our results reveal notable declines in detection accuracy for rephrased emails across all detectors, highlighting critical weaknesses in current phishing defenses.
  arXiv  Detail & Related papers  (2024-11-21T06:20:29Z)
• AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning [93.77763753231338]
  Adversarial Contrastive Prompt Tuning (ACPT) is proposed to fine-tune the CLIP image encoder to extract similar embeddings for any two intermediate adversarial queries. We show that ACPT can detect 7 state-of-the-art query-based attacks with $>99%$ detection rate within 5 shots. We also show that ACPT is robust to 3 types of adaptive attacks.
  arXiv  Detail & Related papers  (2024-08-04T09:53:50Z)
• An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach [2.8282906214258805]
  Phishing email is a serious cyber threat that tries to deceive users by sending false emails with the intention of stealing confidential information or causing financial harm. Despite extensive academic research, phishing detection remains an ongoing and formidable challenge in the cybersecurity landscape. We present an optimized, fine-tuned transformer-based DistilBERT model designed for the detection of phishing emails.
  arXiv  Detail & Related papers  (2024-02-21T15:23:21Z)
• Prompted Contextual Vectors for Spear-Phishing Detection [41.26408609344205]
  Spear-phishing attacks present a significant security challenge.<n>We propose a detection approach based on a novel document vectorization method.<n>Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails.
  arXiv  Detail & Related papers  (2024-02-13T09:12:55Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Verifying the Robustness of Automatic Credibility Assessment [50.55687778699995]
  We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• Robust and Verifiable Information Embedding Attacks to Deep Neural Networks via Error-Correcting Codes [81.85509264573948]
  In the era of deep learning, a user often leverages a third-party machine learning tool to train a deep neural network (DNN) classifier. In an information embedding attack, an attacker is the provider of a malicious third-party machine learning tool. In this work, we aim to design information embedding attacks that are verifiable and robust against popular post-processing methods.
  arXiv  Detail & Related papers  (2020-10-26T17:42:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.