SpearBot: Leveraging Large Language Models in a Generative-Critique Framework for Spear-Phishing Email Generation

• URL: http://arxiv.org/abs/2412.11109v1
• Date: Sun, 15 Dec 2024 08:13:12 GMT
• Title: SpearBot: Leveraging Large Language Models in a Generative-Critique Framework for Spear-Phishing Email Generation
• Authors: Qinglin Qi, Yun Luo, Yijia Xu, Wenbo Guo, Yong Fang,
• Abstract summary: Large Language Models (LLMs) are increasingly capable, aiding in tasks such as content generation. This paper proposes an adversarial framework, SpearBot, to generate spear-phishing emails.
• Score: 9.322176274376774
• License:
• Abstract: Large Language Models (LLMs) are increasingly capable, aiding in tasks such as content generation, yet they also pose risks, particularly in generating harmful spear-phishing emails. These emails, crafted to entice clicks on malicious URLs, threaten personal information security. This paper proposes an adversarial framework, SpearBot, which utilizes LLMs to generate spear-phishing emails with various phishing strategies. Through specifically crafted jailbreak prompts, SpearBot circumvents security policies and introduces other LLM instances as critics. When a phishing email is identified by the critic, SpearBot refines the generated email based on the critique feedback until it can no longer be recognized as phishing, thereby enhancing its deceptive quality. To evaluate the effectiveness of SpearBot, we implement various machine-based defenders and assess how well the phishing emails generated could deceive them. Results show these emails often evade detection to a large extent, underscoring their deceptive quality. Additionally, human evaluations of the emails' readability and deception are conducted through questionnaires, confirming their convincing nature and the significant potential harm of the generated phishing emails.

Related papers

• APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users [2.3618982787621]
  Large Language Models (LLMs) offer significant promise for text processing in various domains. We present APOLLO, a tool based on OpenAI's GPT-4o to detect phishing emails and generate explanation messages. We also conducted a study with 20 participants, comparing four different explanations presented as phishing warnings.
  arXiv  Detail & Related papers  (2024-10-10T14:53:39Z)
• Eyes on the Phish(er): Towards Understanding Users' Email Processing Pattern and Mental Models in Phishing Detection [0.4543820534430522]
  This study examines how workload affects susceptibility to phishing. We use eye-tracking technology to observe participants' reading patterns and interactions with phishing emails. Our results provide concrete evidence that attention to the email sender can reduce phishing susceptibility.
  arXiv  Detail & Related papers  (2024-09-12T02:57:49Z)
• BaThe: Defense against the Jailbreak Attack in Multimodal Large Language Models by Treating Harmful Instruction as Backdoor Trigger [67.75420257197186]
  In this work, we propose $textbfBaThe, a simple yet effective jailbreak defense mechanism. Jailbreak backdoor attack uses harmful instructions combined with manually crafted strings as triggers to make the backdoored model generate prohibited responses. We assume that harmful instructions can function as triggers, and if we alternatively set rejection responses as the triggered response, the backdoored model then can defend against jailbreak attacks.
  arXiv  Detail & Related papers  (2024-08-17T04:43:26Z)
• Analysis and prevention of AI-based phishing email attacks [0.0]
  generative AI can be used to send each potential victim a different email. We use different machine learning tools to test the ability of automatic text analysis to identify AI-generated phishing emails. The paper also describes the corpus of AI-generated phishing emails that is made open to the public.
  arXiv  Detail & Related papers  (2024-05-08T21:40:49Z)
• Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts [2.6012482282204004]
  Phishing, a prevalent cybercrime tactic for decades, remains a significant threat in today's digital world. This paper aims to analyze the effectiveness of 15 Large Language Models (LLMs) in detecting phishing attempts.
  arXiv  Detail & Related papers  (2024-04-23T19:55:18Z)
• ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection [2.3999111269325266]
  This study introduces ChatSpamDetector, a system that uses large language models (LLMs) to detect phishing emails. By converting email data into a prompt suitable for LLM analysis, the system provides a highly accurate determination of whether an email is phishing or not. We conducted an evaluation using a comprehensive phishing email dataset and compared our system to several LLMs and baseline systems.
  arXiv  Detail & Related papers  (2024-02-28T06:28:15Z)
• Prompted Contextual Vectors for Spear-Phishing Detection [41.26408609344205]
  Spear-phishing attacks present a significant security challenge. We propose a detection approach based on a novel document vectorization method. Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails.
  arXiv  Detail & Related papers  (2024-02-13T09:12:55Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)
• Deep convolutional forest: a dynamic deep ensemble approach for spam detection in text [219.15486286590016]
  This paper introduces a dynamic deep ensemble model for spam detection that adjusts its complexity and extracts features automatically. As a result, the model achieved high precision, recall, f1-score and accuracy of 98.38%.
  arXiv  Detail & Related papers  (2021-10-10T17:19:37Z)
• Robust and Verifiable Information Embedding Attacks to Deep Neural Networks via Error-Correcting Codes [81.85509264573948]
  In the era of deep learning, a user often leverages a third-party machine learning tool to train a deep neural network (DNN) classifier. In an information embedding attack, an attacker is the provider of a malicious third-party machine learning tool. In this work, we aim to design information embedding attacks that are verifiable and robust against popular post-processing methods.
  arXiv  Detail & Related papers  (2020-10-26T17:42:42Z)
• Phishing and Spear Phishing: examples in Cyber Espionage and techniques to protect against them [91.3755431537592]
  Phishing attacks have become the most used technique in the online scams, initiating more than 91% of cyberattacks, from 2012 onwards. This study reviews how Phishing and Spear Phishing attacks are carried out by the phishers, through 5 steps which magnify the outcome.
  arXiv  Detail & Related papers  (2020-05-31T18:10:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.