Prompted Contextual Vectors for Spear-Phishing Detection

• URL: http://arxiv.org/abs/2402.08309v2
• Date: Wed, 14 Feb 2024 08:10:38 GMT
• Title: Prompted Contextual Vectors for Spear-Phishing Detection
• Authors: Daniel Nahmias, Gal Engelberg, Dan Klein, Asaf Shabtai
• Abstract summary: Spear-phishing attacks present a significant security challenge. We propose a detection approach based on a novel document vectorization method. Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails.
• Score: 45.07804966535239
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Spear-phishing attacks present a significant security challenge, with large language models (LLMs) escalating the threat by generating convincing emails and facilitating target reconnaissance. To address this, we propose a detection approach based on a novel document vectorization method that utilizes an ensemble of LLMs to create representation vectors. By prompting LLMs to reason and respond to human-crafted questions, we quantify the presence of common persuasion principles in the email's content, producing prompted contextual document vectors for a downstream supervised machine learning model. We evaluate our method using a unique dataset generated by a proprietary system that automates target reconnaissance and spear-phishing email creation. Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails, with the training set comprising only traditional phishing and benign emails. Key contributions include an innovative document vectorization method utilizing LLM reasoning, a publicly available dataset of high-quality spear-phishing emails, and the demonstrated effectiveness of our method in detecting such emails. This methodology can be utilized for various document classification tasks, particularly in adversarial problem domains.

Related papers

• Attention Tracker: Detecting Prompt Injection Attacks in LLMs [62.247841717696765]
  Large Language Models (LLMs) have revolutionized various domains but remain vulnerable to prompt injection attacks. We introduce the concept of the distraction effect, where specific attention heads shift focus from the original instruction to the injected instruction. We propose Attention Tracker, a training-free detection method that tracks attention patterns on instruction to detect prompt injection attacks.
  arXiv  Detail & Related papers  (2024-11-01T04:05:59Z)
• Evaluating Large Language Model based Personal Information Extraction and Countermeasures [63.91918057570824]
  Large language model (LLM) can be misused by attackers to accurately extract various personal information from personal profiles. LLM outperforms conventional methods at such extraction. prompt injection can mitigate such risk to a large extent and outperforms conventional countermeasures.
  arXiv  Detail & Related papers  (2024-08-14T04:49:30Z)
• Are you still on track!? Catching LLM Task Drift with Activations [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users. We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set. We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Detecting Scams Using Large Language Models [19.7220607313348]
  Large Language Models (LLMs) have gained prominence in various applications, including security. This paper explores the utility of LLMs in scam detection, a critical aspect of cybersecurity. We propose a novel use case for LLMs to identify scams, such as phishing, advance fee fraud, and romance scams.
  arXiv  Detail & Related papers  (2024-02-05T16:13:54Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Profiler: Profile-Based Model to Detect Phishing Emails [15.109679047753355]
  We propose a multidimensional risk assessment of emails to reduce the feasibility of an attacker adapting their email and avoiding detection. We develop a risk assessment framework that includes three models which analyse an email's (1) threat level, (2) cognitive manipulation, and (3) email type. Our Profiler can be used in conjunction with ML approaches, to reduce their misclassifications or as a labeller for large email data sets in the training stage.
  arXiv  Detail & Related papers  (2022-08-18T10:01:55Z)
• Anomaly Detection in Emails using Machine Learning and Header Information [0.0]
  Anomalies in emails such as phishing and spam present major security risks. Previous studies on email anomaly detection relied on a single type of anomaly and the analysis of the email body and subject content. This study conducted feature extraction and selection on email header datasets and leveraged both multi and one-class anomaly detection approaches.
  arXiv  Detail & Related papers  (2022-03-19T23:31:23Z)
• Phishing Detection through Email Embeddings [2.099922236065961]
  The problem of detecting phishing emails through machine learning techniques has been discussed extensively in the literature. In this paper, we crafted a set of phishing and legitimate emails with similar indicators in order to investigate whether these cues are captured or disregarded by email embeddings. Our results show that using these indicators, email embeddings techniques is effective for classifying emails as phishing or legitimate.
  arXiv  Detail & Related papers  (2020-12-28T21:16:41Z)
• Detection of Adversarial Supports in Few-shot Classifiers Using Feature Preserving Autoencoders and Self-Similarity [89.26308254637702]
  We propose a detection strategy to highlight adversarial support sets. We make use of feature preserving autoencoder filtering and also the concept of self-similarity of a support set to perform this detection. Our method is attack-agnostic and also the first to explore detection for few-shot classifiers to the best of our knowledge.
  arXiv  Detail & Related papers  (2020-12-09T14:13:41Z)
• Modeling Coherency in Generated Emails by Leveraging Deep Neural Learners [6.891238879512674]
  Advanced machine learning and natural language techniques enable attackers to launch sophisticated and targeted social engineering-based attacks. Email masquerading using targeted emails to fool the victim is an advanced attack method. We demonstrate the generation of short and targeted text messages using the deep model.
  arXiv  Detail & Related papers  (2020-07-14T23:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.