ExplainableDetector: Exploring Transformer-based Language Modeling Approach for SMS Spam Detection with Explainability Analysis

• URL: http://arxiv.org/abs/2405.08026v1
• Date: Sun, 12 May 2024 11:42:05 GMT
• Title: ExplainableDetector: Exploring Transformer-based Language Modeling Approach for SMS Spam Detection with Explainability Analysis
• Authors: Mohammad Amaz Uddin, Muhammad Nazrul Islam, Leandros Maglaras, Helge Janicke, Iqbal H. Sarker,
• Abstract summary: The number of SMS spam has expanded significantly in recent years. The unstructured format of SMS data creates significant challenges for SMS spam detection. We employ optimized and fine-tuned transformer-based Large Language Models (LLMs) to solve the problem of spam message detection.
• Score: 2.849988619791745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: SMS, or short messaging service, is a widely used and cost-effective communication medium that has sadly turned into a haven for unwanted messages, commonly known as SMS spam. With the rapid adoption of smartphones and Internet connectivity, SMS spam has emerged as a prevalent threat. Spammers have taken notice of the significance of SMS for mobile phone users. Consequently, with the emergence of new cybersecurity threats, the number of SMS spam has expanded significantly in recent years. The unstructured format of SMS data creates significant challenges for SMS spam detection, making it more difficult to successfully fight spam attacks in the cybersecurity domain. In this work, we employ optimized and fine-tuned transformer-based Large Language Models (LLMs) to solve the problem of spam message detection. We use a benchmark SMS spam dataset for this spam detection and utilize several preprocessing techniques to get clean and noise-free data and solve the class imbalance problem using the text augmentation technique. The overall experiment showed that our optimized fine-tuned BERT (Bidirectional Encoder Representations from Transformers) variant model RoBERTa obtained high accuracy with 99.84\%. We also work with Explainable Artificial Intelligence (XAI) techniques to calculate the positive and negative coefficient scores which explore and explain the fine-tuned model transparency in this text-based spam SMS detection task. In addition, traditional Machine Learning (ML) models were also examined to compare their performance with the transformer-based models. This analysis describes how LLMs can make a good impact on complex textual-based spam data in the cybersecurity field.

Related papers

• Vulnerability of LLMs to Vertically Aligned Text Manipulations [108.6908427615402]
  Large language models (LLMs) have become highly effective at performing text classification tasks. modifying input formats, such as vertically aligning words for encoder-based models, can substantially lower accuracy in text classification tasks. Do decoder-based LLMs exhibit similar vulnerabilities to vertically formatted text input?
  arXiv  Detail & Related papers  (2024-10-26T00:16:08Z)
• SMS Spam Detection and Classification to Combat Abuse in Telephone Networks Using Natural Language Processing [0.0]
  This research addresses the pervasive issue of SMS spam, which poses threats to users' privacy and security. The study introduces a novel approach utilizing Natural Language Processing (NLP) and machine learning models, particularly BERT (Bidirectional Representations from Transformers) for spam detection and classification. Evaluation results revealed that the Na"ive Bayes + BERT model achieves the highest accuracy at 97.31% with the fastest execution time of 0.3 seconds on the test dataset.
  arXiv  Detail & Related papers  (2024-06-04T13:44:36Z)
• Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore [51.65730053591696]
  We propose a simple but effective black-box zero-shot detection approach. It is predicated on the observation that human-written texts typically contain more grammatical errors than LLM-generated texts. Our method achieves an average AUROC of 98.7% and shows strong robustness against paraphrase and adversarial perturbation attacks.
  arXiv  Detail & Related papers  (2024-05-07T12:57:01Z)
• SpamDam: Towards Privacy-Preserving and Adversary-Resistant SMS Spam Detection [2.0355793807035094]
  SpamDam is a SMS spam detection framework designed to overcome key challenges in detecting and understanding SMS spam. We have compiled over 76K SMS spam messages from Twitter and Weibo between 2018 and 2023, forming the largest dataset of its kind. We have rigorously tested the adversarial robustness of SMS spam detection models, introducing the novel reverse backdoor attack.
  arXiv  Detail & Related papers  (2024-04-15T06:07:10Z)
• Evaluating the Performance of ChatGPT for Spam Email Detection [9.585304538597414]
  This study attempts to evaluate ChatGPT's capabilities for spam identification in both English and Chinese email datasets. We employ ChatGPT for spam email detection using in-context learning, which requires a prompt instruction and a few demonstrations. We also investigate how the number of demonstrations in the prompt affects the performance of ChatGPT.
  arXiv  Detail & Related papers  (2024-02-23T04:52:08Z)
• MGTBench: Benchmarking Machine-Generated Text Detection [54.81446366272403]
  This paper proposes the first benchmark framework for MGT detection against powerful large language models (LLMs) We show that a larger number of words in general leads to better performance and most detection methods can achieve similar performance with much fewer training samples. Our findings indicate that the model-based detection methods still perform well in the text attribution task.
  arXiv  Detail & Related papers  (2023-03-26T21:12:36Z)
• Spam Detection Using BERT [0.0]
  We build a spam detector using BERT pre-trained model that classifies emails and messages by understanding to their context. Our spam detector performance was 98.62%, 97.83%, 99.13% and 99.28% respectively.
  arXiv  Detail & Related papers  (2022-06-06T09:09:40Z)
• Deep convolutional forest: a dynamic deep ensemble approach for spam detection in text [219.15486286590016]
  This paper introduces a dynamic deep ensemble model for spam detection that adjusts its complexity and extracts features automatically. As a result, the model achieved high precision, recall, f1-score and accuracy of 98.38%.
  arXiv  Detail & Related papers  (2021-10-10T17:19:37Z)
• MOST: A Multi-Oriented Scene Text Detector with Localization Refinement [67.35280008722255]
  We propose a new algorithm for scene text detection, which puts forward a set of strategies to significantly improve the quality of text localization. Specifically, a Text Feature Alignment Module (TFAM) is proposed to dynamically adjust the receptive fields of features. A Position-Aware Non-Maximum Suppression (PA-NMS) module is devised to exclude unreliable ones.
  arXiv  Detail & Related papers  (2021-04-02T14:34:41Z)
• Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding [80.3811072650087]
  We study natural language watermarking as a defense to help better mark and trace the provenance of text. We introduce the Adversarial Watermarking Transformer (AWT) with a jointly trained encoder-decoder and adversarial training. AWT is the first end-to-end model to hide data in text by automatically learning -- without ground truth -- word substitutions along with their locations.
  arXiv  Detail & Related papers  (2020-09-07T11:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.