E-PhishGen: Unlocking Novel Research in Phishing Email Detection

• URL: http://arxiv.org/abs/2509.01791v2
• Date: Mon, 15 Sep 2025 13:13:33 GMT
• Title: E-PhishGen: Unlocking Novel Research in Phishing Email Detection
• Authors: Luca Pajola, Eugenio Caripoti, Stefan Banzer, Simeone Pizzi, Mauro Conti, Giovanni Apruzzese,
• Abstract summary: This "open problem" paper carries out a critical assessment of scientific works in the context of phishing email detection.<n>We show that phishing email detection is still an open problem -- and provide the means to tackle such a problem by future research.
• Score: 17.071710380823003
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Every day, our inboxes are flooded with unsolicited emails, ranging between annoying spam to more subtle phishing scams. Unfortunately, despite abundant prior efforts proposing solutions achieving near-perfect accuracy, the reality is that countering malicious emails still remains an unsolved dilemma. This "open problem" paper carries out a critical assessment of scientific works in the context of phishing email detection. First, we focus on the benchmark datasets that have been used to assess the methods proposed in research. We find that most prior work relied on datasets containing emails that -- we argue -- are not representative of current trends, and mostly encompass the English language. Based on this finding, we then re-implement and re-assess a variety of detection methods reliant on machine learning (ML), including large-language models (LLM), and release all of our codebase -- an (unfortunately) uncommon practice in related research. We show that most such methods achieve near-perfect performance when trained and tested on the same dataset -- a result which intrinsically hinders development (how can future research outperform methods that are already near perfect?). To foster the creation of "more challenging benchmarks" that reflect current phishing trends, we propose E-PhishGEN, an LLM-based (and privacy-savvy) framework to generate novel phishing-email datasets. We use our E-PhishGEN to create E-PhishLLM, a novel phishing-email detection dataset containing 16616 emails in three languages. We use E-PhishLLM to test the detectors we considered, showing a much lower performance than that achieved on existing benchmarks -- indicating a larger room for improvement. We also validate the quality of E-PhishLLM with a user study (n=30). To sum up, we show that phishing email detection is still an open problem -- and provide the means to tackle such a problem by future research.

Related papers

• Constructing and Benchmarking: a Labeled Email Dataset for Text-Based Phishing and Spam Detection Framework [0.37687375904925485]
  This study presents a comprehensive email dataset containing phishing, spam, and legitimate messages.<n>Each email is annotated with its category, emotional appeal, authority, and underlying motivation.<n>Results highlight strong phishing detection capabilities but reveal persistent challenges in distinguishing spam from legitimate emails.
  arXiv  Detail & Related papers  (2025-11-26T14:40:06Z)
• Robust ML-based Detection of Conventional, LLM-Generated, and Adversarial Phishing Emails Using Advanced Text Preprocessing [3.3166006294048427]
  We propose a robust phishing email detection system featuring an enhanced text preprocessing pipeline.<n>Our approach integrates widely adopted natural language processing (NLP) feature extraction techniques and machine learning algorithms.<n>We evaluate our models on publicly available datasets comprising both phishing and legitimate emails, achieving a detection accuracy of 94.26% and F1-score of 84.39%.
  arXiv  Detail & Related papers  (2025-10-13T20:34:19Z)
• Leave No TRACE: Black-box Detection of Copyrighted Dataset Usage in Large Language Models via Watermarking [51.74368870268278]
  We propose TRACE, a framework for fully black-box detection of copyrighted dataset usage in large language models.<n>textttTRACE rewrites datasets with distortion-free watermarks guided by a private key.<n>Across diverse datasets and model families, TRACE consistently achieves significant detections.
  arXiv  Detail & Related papers  (2025-10-03T12:53:02Z)
• MeAJOR Corpus: A Multi-Source Dataset for Phishing Email Detection [0.0]
  This paper presents MeAJOR, a novel, multi-source phishing email dataset.<n>It integrates 135894 samples representing a broad number of phishing tactics and legitimate emails.<n>By integrating broad features from multiple categories, our dataset provides a reusable and consistent resource.
  arXiv  Detail & Related papers  (2025-07-23T22:57:08Z)
• Enhancing Phishing Email Identification with Large Language Models [0.40792653193642503]
  We study the efficacy of large language models (LLMs) in detecting phishing emails.<n>Experiments show that the LLM achieves a high accuracy rate at high precision.
  arXiv  Detail & Related papers  (2025-02-07T08:45:50Z)
• Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method [108.56493934296687]
  We introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection.<n>We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text.
  arXiv  Detail & Related papers  (2024-09-23T07:55:35Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users.<n>We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set.<n>We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Prompted Contextual Vectors for Spear-Phishing Detection [41.26408609344205]
  Spear-phishing attacks present a significant security challenge.<n>We propose a detection approach based on a novel document vectorization method.<n>Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails.
  arXiv  Detail & Related papers  (2024-02-13T09:12:55Z)
• Profiler: Profile-Based Model to Detect Phishing Emails [15.109679047753355]
  We propose a multidimensional risk assessment of emails to reduce the feasibility of an attacker adapting their email and avoiding detection. We develop a risk assessment framework that includes three models which analyse an email's (1) threat level, (2) cognitive manipulation, and (3) email type. Our Profiler can be used in conjunction with ML approaches, to reduce their misclassifications or as a labeller for large email data sets in the training stage.
  arXiv  Detail & Related papers  (2022-08-18T10:01:55Z)
• Black-box Dataset Ownership Verification via Backdoor Watermarking [67.69308278379957]
  We formulate the protection of released datasets as verifying whether they are adopted for training a (suspicious) third-party model. We propose to embed external patterns via backdoor watermarking for the ownership verification to protect them. Specifically, we exploit poison-only backdoor attacks ($e.g.$, BadNets) for dataset watermarking and design a hypothesis-test-guided method for dataset verification.
  arXiv  Detail & Related papers  (2022-08-04T05:32:20Z)
• Anomaly Detection in Emails using Machine Learning and Header Information [0.0]
  Anomalies in emails such as phishing and spam present major security risks. Previous studies on email anomaly detection relied on a single type of anomaly and the analysis of the email body and subject content. This study conducted feature extraction and selection on email header datasets and leveraged both multi and one-class anomaly detection approaches.
  arXiv  Detail & Related papers  (2022-03-19T23:31:23Z)
• Phishing Detection through Email Embeddings [2.099922236065961]
  The problem of detecting phishing emails through machine learning techniques has been discussed extensively in the literature. In this paper, we crafted a set of phishing and legitimate emails with similar indicators in order to investigate whether these cues are captured or disregarded by email embeddings. Our results show that using these indicators, email embeddings techniques is effective for classifying emails as phishing or legitimate.
  arXiv  Detail & Related papers  (2020-12-28T21:16:41Z)
• Learning with Weak Supervision for Email Intent Detection [56.71599262462638]
  We propose to leverage user actions as a source of weak supervision to detect intents in emails. We develop an end-to-end robust deep neural network model for email intent identification.
  arXiv  Detail & Related papers  (2020-05-26T23:41:05Z)
• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking [92.48078680697311]
  Multi-object tracking (MOT) is an important problem in computer vision. We present a simple yet effective approach termed as FairMOT based on the anchor-free object detection architecture CenterNet. The approach achieves high accuracy for both detection and tracking.
  arXiv  Detail & Related papers  (2020-04-04T08:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.