Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts

• URL: http://arxiv.org/abs/2404.15485v3
• Date: Thu, 6 Jun 2024 21:03:03 GMT
• Title: Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts
• Authors: Het Patel, Umair Rehman, Farkhund Iqbal,
• Abstract summary: Phishing, a prevalent cybercrime tactic for decades, remains a significant threat in today's digital world. This paper aims to analyze the effectiveness of 15 Large Language Models (LLMs) in detecting phishing attempts.
• Score: 2.6012482282204004
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Phishing, a prevalent cybercrime tactic for decades, remains a significant threat in today's digital world. By leveraging clever social engineering elements and modern technology, cybercrime targets many individuals, businesses, and organizations to exploit trust and security. These cyber-attackers are often disguised in many trustworthy forms to appear as legitimate sources. By cleverly using psychological elements like urgency, fear, social proof, and other manipulative strategies, phishers can lure individuals into revealing sensitive and personalized information. Building on this pervasive issue within modern technology, this paper aims to analyze the effectiveness of 15 Large Language Models (LLMs) in detecting phishing attempts, specifically focusing on a randomized set of "419 Scam" emails. The objective is to determine which LLMs can accurately detect phishing emails by analyzing a text file containing email metadata based on predefined criteria. The experiment concluded that the following models, ChatGPT 3.5, GPT-3.5-Turbo-Instruct, and ChatGPT, were the most effective in detecting phishing emails.

Related papers

• Next-Generation Phishing: How LLM Agents Empower Cyber Attackers [10.067883724547182]
  The escalating threat of phishing emails has become increasingly sophisticated with the rise of Large Language Models (LLMs) As attackers exploit LLMs to craft more convincing and evasive phishing emails, it is crucial to assess the resilience of current phishing defenses. We conduct a comprehensive evaluation of traditional phishing detectors, such as Gmail Spam Filter, Apache SpamAssassin, and Proofpoint, as well as machine learning models like SVM, Logistic Regression, and Naive Bayes. Our results reveal notable declines in detection accuracy for rephrased emails across all detectors, highlighting critical weaknesses in current phishing defenses.
  arXiv  Detail & Related papers  (2024-11-21T06:20:29Z)
• APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users [2.3618982787621]
  Large Language Models (LLMs) offer significant promise for text processing in various domains. We present APOLLO, a tool based on OpenAI's GPT-4o to detect phishing emails and generate explanation messages. We also conducted a study with 20 participants, comparing four different explanations presented as phishing warnings.
  arXiv  Detail & Related papers  (2024-10-10T14:53:39Z)
• ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection [2.3999111269325266]
  This study introduces ChatSpamDetector, a system that uses large language models (LLMs) to detect phishing emails. By converting email data into a prompt suitable for LLM analysis, the system provides a highly accurate determination of whether an email is phishing or not. We conducted an evaluation using a comprehensive phishing email dataset and compared our system to several LLMs and baseline systems.
  arXiv  Detail & Related papers  (2024-02-28T06:28:15Z)
• An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach [2.8282906214258805]
  Phishing email is a serious cyber threat that tries to deceive users by sending false emails with the intention of stealing confidential information or causing financial harm. Despite extensive academic research, phishing detection remains an ongoing and formidable challenge in the cybersecurity landscape. We present an optimized, fine-tuned transformer-based DistilBERT model designed for the detection of phishing emails.
  arXiv  Detail & Related papers  (2024-02-21T15:23:21Z)
• Prompted Contextual Vectors for Spear-Phishing Detection [45.07804966535239]
  Spear-phishing attacks present a significant security challenge. We propose a detection approach based on a novel document vectorization method. Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails.
  arXiv  Detail & Related papers  (2024-02-13T09:12:55Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• SoK: Human-Centered Phishing Susceptibility [4.794822439017277]
  We propose a three-stage Phishing Susceptibility Model (PSM) for explaining how humans are involved in phishing detection and prevention. This model reveals several research gaps that need to be addressed to improve users' detection performance.
  arXiv  Detail & Related papers  (2022-02-16T07:26:53Z)
• Deep convolutional forest: a dynamic deep ensemble approach for spam detection in text [219.15486286590016]
  This paper introduces a dynamic deep ensemble model for spam detection that adjusts its complexity and extracts features automatically. As a result, the model achieved high precision, recall, f1-score and accuracy of 98.38%.
  arXiv  Detail & Related papers  (2021-10-10T17:19:37Z)
• Robust and Verifiable Information Embedding Attacks to Deep Neural Networks via Error-Correcting Codes [81.85509264573948]
  In the era of deep learning, a user often leverages a third-party machine learning tool to train a deep neural network (DNN) classifier. In an information embedding attack, an attacker is the provider of a malicious third-party machine learning tool. In this work, we aim to design information embedding attacks that are verifiable and robust against popular post-processing methods.
  arXiv  Detail & Related papers  (2020-10-26T17:42:42Z)
• Phishing and Spear Phishing: examples in Cyber Espionage and techniques to protect against them [91.3755431537592]
  Phishing attacks have become the most used technique in the online scams, initiating more than 91% of cyberattacks, from 2012 onwards. This study reviews how Phishing and Spear Phishing attacks are carried out by the phishers, through 5 steps which magnify the outcome.
  arXiv  Detail & Related papers  (2020-05-31T18:10:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.