Evaluating Large Language Models' Capability to Launch Fully Automated Spear Phishing Campaigns: Validated on Human Subjects

• URL: http://arxiv.org/abs/2412.00586v1
• Date: Sat, 30 Nov 2024 21:02:06 GMT
• Title: Evaluating Large Language Models' Capability to Launch Fully Automated Spear Phishing Campaigns: Validated on Human Subjects
• Authors: Fred Heiding, Simon Lermen, Andrew Kao, Bruce Schneier, Arun Vishwanath,
• Abstract summary: We evaluate the capability of large language models to conduct personalized phishing attacks. We compare their performance with human experts and AI models from last year.
• Score: 0.0699049312989311
• License:
• Abstract: In this paper, we evaluate the capability of large language models to conduct personalized phishing attacks and compare their performance with human experts and AI models from last year. We include four email groups with a combined total of 101 participants: A control group of arbitrary phishing emails, which received a click-through rate (recipient pressed a link in the email) of 12%, emails generated by human experts (54% click-through), fully AI-automated emails 54% (click-through), and AI emails utilizing a human-in-the-loop (56% click-through). Thus, the AI-automated attacks performed on par with human experts and 350% better than the control group. The results are a significant improvement from similar studies conducted last year, highlighting the increased deceptive capabilities of AI models. Our AI-automated emails were sent using a custom-built tool that automates the entire spear phishing process, including information gathering and creating personalized vulnerability profiles for each target. The AI-gathered information was accurate and useful in 88% of cases and only produced inaccurate profiles for 4% of the participants. We also use language models to detect the intention of emails. Claude 3.5 Sonnet scored well above 90% with low false-positive rates and detected several seemingly benign emails that passed human detection. Lastly, we analyze the economics of phishing, highlighting how AI enables attackers to target more individuals at lower cost and increase profitability by up to 50 times for larger audiences.

Related papers

• APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users [2.3618982787621]
  Large Language Models (LLMs) offer significant promise for text processing in various domains. We present APOLLO, a tool based on OpenAI's GPT-4o to detect phishing emails and generate explanation messages. We also conducted a study with 20 participants, comparing four different explanations presented as phishing warnings.
  arXiv  Detail & Related papers  (2024-10-10T14:53:39Z)
• Analysis and prevention of AI-based phishing email attacks [0.0]
  generative AI can be used to send each potential victim a different email. We use different machine learning tools to test the ability of automatic text analysis to identify AI-generated phishing emails. The paper also describes the corpus of AI-generated phishing emails that is made open to the public.
  arXiv  Detail & Related papers  (2024-05-08T21:40:49Z)
• Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts [2.6012482282204004]
  Phishing, a prevalent cybercrime tactic for decades, remains a significant threat in today's digital world. This paper aims to analyze the effectiveness of 15 Large Language Models (LLMs) in detecting phishing attempts.
  arXiv  Detail & Related papers  (2024-04-23T19:55:18Z)
• A LLM Assisted Exploitation of AI-Guardian [57.572998144258705]
  We evaluate the robustness of AI-Guardian, a recent defense to adversarial examples published at IEEE S&P 2023. We write none of the code to attack this model, and instead prompt GPT-4 to implement all attack algorithms following our instructions and guidance. This process was surprisingly effective and efficient, with the language model at times producing code from ambiguous instructions faster than the author of this paper could have done.
  arXiv  Detail & Related papers  (2023-07-20T17:33:25Z)
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense [56.077252790310176]
  We present a paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking. We introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.
  arXiv  Detail & Related papers  (2023-03-23T16:29:27Z)
• Can AI-Generated Text be Reliably Detected? [50.95804851595018]
  Large Language Models (LLMs) perform impressively well in various applications. The potential for misuse of these models in activities such as plagiarism, generating fake news, and spamming has raised concern about their responsible use. We stress-test the robustness of these AI text detectors in the presence of an attacker.
  arXiv  Detail & Related papers  (2023-03-17T17:53:19Z)
• Profiler: Profile-Based Model to Detect Phishing Emails [15.109679047753355]
  We propose a multidimensional risk assessment of emails to reduce the feasibility of an attacker adapting their email and avoiding detection. We develop a risk assessment framework that includes three models which analyse an email's (1) threat level, (2) cognitive manipulation, and (3) email type. Our Profiler can be used in conjunction with ML approaches, to reduce their misclassifications or as a labeller for large email data sets in the training stage.
  arXiv  Detail & Related papers  (2022-08-18T10:01:55Z)
• Email Summarization to Assist Users in Phishing Identification [1.433758865948252]
  Cyber-phishing attacks are more precise, targeted, and tailored by training data to activate only in the presence of specific information or cues. This work leverages transformer-based machine learning to analyze prospective psychological triggers. We then amalgamate this information and present it to the user to allow them to (i) easily decide whether the email is "phishy" and (ii) self-learn advanced malicious patterns.
  arXiv  Detail & Related papers  (2022-03-24T23:03:46Z)
• Deep convolutional forest: a dynamic deep ensemble approach for spam detection in text [219.15486286590016]
  This paper introduces a dynamic deep ensemble model for spam detection that adjusts its complexity and extracts features automatically. As a result, the model achieved high precision, recall, f1-score and accuracy of 98.38%.
  arXiv  Detail & Related papers  (2021-10-10T17:19:37Z)
• The Threat of Offensive AI to Organizations [52.011307264694665]
  This survey explores the threat of offensive AI on organizations. First, we discuss how AI changes the adversary's methods, strategies, goals, and overall attack model. Then, through a literature review, we identify 33 offensive AI capabilities which adversaries can use to enhance their attacks.
  arXiv  Detail & Related papers  (2021-06-30T01:03:28Z)
• Learning with Weak Supervision for Email Intent Detection [56.71599262462638]
  We propose to leverage user actions as a source of weak supervision to detect intents in emails. We develop an end-to-end robust deep neural network model for email intent identification.
  arXiv  Detail & Related papers  (2020-05-26T23:41:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.