A Comparison of Vulnerability Feature Extraction Methods from Textual Attack Patterns

• URL: http://arxiv.org/abs/2407.06753v2
• Date: Thu, 11 Jul 2024 11:31:15 GMT
• Title: A Comparison of Vulnerability Feature Extraction Methods from Textual Attack Patterns
• Authors: Refat Othman, Bruno Rossi, Russo Barbara,
• Abstract summary: This paper aims to aid cybersecurity researchers and practitioners in choosing attack extraction methods. Term Frequency-Inverse Document Frequency (TF-IDF) outperforms the other four methods with a precision of 75% and an F1 score of 64%.
• Score: 0.22940141855172028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, threat reports from cybersecurity vendors incorporate detailed descriptions of attacks within unstructured text. Knowing vulnerabilities that are related to these reports helps cybersecurity researchers and practitioners understand and adjust to evolving attacks and develop mitigation plans. This paper aims to aid cybersecurity researchers and practitioners in choosing attack extraction methods to enhance the monitoring and sharing of threat intelligence. In this work, we examine five feature extraction methods (TF-IDF, LSI, BERT, MiniLM, RoBERTa) and find that Term Frequency-Inverse Document Frequency (TF-IDF) outperforms the other four methods with a precision of 75\% and an F1 score of 64\%. The findings offer valuable insights to the cybersecurity community, and our research can aid cybersecurity researchers in evaluating and comparing the effectiveness of upcoming extraction methods.

Related papers

• CTISum: A New Benchmark Dataset For Cyber Threat Intelligence Summarization [14.287652216484863]
  We present CTISum, a new benchmark for CTI summarization task. Considering the importance of attack process, a novel fine-grained subtask of attack process summarization is proposed.
  arXiv  Detail & Related papers  (2024-08-13T02:25:16Z)
• EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection [19.885698402507145]
  Adversarial examples can exploit vulnerabilities within deep neural networks. This study showcases the susceptibility of deep learning models to adversarial attacks, which can achieve 100% attack success rate.
  arXiv  Detail & Related papers  (2024-07-27T09:04:54Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• A Relevance Model for Threat-Centric Ranking of Cybersecurity Vulnerabilities [0.29998889086656577]
  The relentless process of tracking and remediating vulnerabilities is a top concern for cybersecurity professionals. We provide a framework for vulnerability management specifically focused on mitigating threats using adversary criteria derived from MITRE ATT&CK. Our results show an average 71.5% - 91.3% improvement towards the identification of vulnerabilities likely to be targeted and exploited by cyber threat actors.
  arXiv  Detail & Related papers  (2024-06-09T23:29:12Z)
• From Threat Reports to Continuous Threat Intelligence: A Comparison of Attack Technique Extraction Methods from Textual Artifacts [11.396560798899412]
  Threat reports contain detailed descriptions of attack Tactics, Techniques, and Procedures (TTP) written in an unstructured text format. TTP extraction methods are proposed in the literature, but not all of these methods are compared to one another or to a baseline. In this work, we identify ten existing TTP extraction studies from the literature and implement five methods from the ten studies. We find two methods, based on Term Frequency-Inverse Document Frequency(TFIDF) and Latent Semantic Indexing (LSI), outperform the other three methods with a F1 score of 84% and 83%,
  arXiv  Detail & Related papers  (2022-10-05T23:21:41Z)
• Towards Automated Classification of Attackers' TTPs by combining NLP with ML Techniques [77.34726150561087]
  We evaluate and compare different Natural Language Processing (NLP) and machine learning techniques used for security information extraction in research. Based on our investigations we propose a data processing pipeline that automatically classifies unstructured text according to attackers' tactics and techniques.
  arXiv  Detail & Related papers  (2022-07-18T09:59:21Z)
• Improving robustness of jet tagging algorithms with adversarial training [56.79800815519762]
  We investigate the vulnerability of flavor tagging algorithms via application of adversarial attacks. We present an adversarial training strategy that mitigates the impact of such simulated attacks.
  arXiv  Detail & Related papers  (2022-03-25T19:57:19Z)
• A Review of Adversarial Attack and Defense for Classification Methods [78.50824774203495]
  This paper focuses on the generation and guarding of adversarial examples. It is the hope of the authors that this paper will encourage more statisticians to work on this important and exciting field of generating and defending against adversarial examples.
  arXiv  Detail & Related papers  (2021-11-18T22:13:43Z)
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis [71.57324258813674]
  This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model against different unsafe inputs. Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs.
  arXiv  Detail & Related papers  (2021-01-28T16:38:26Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)
• Automated Retrieval of ATT&CK Tactics and Techniques for Cyber Threat Reports [5.789368942487406]
  We evaluate several classification approaches to automatically retrieve Tactics, Techniques and Procedures from unstructured text. We present rcATT, a tool built on top of our findings and freely distributed to the security community to support cyber threat report automated analysis.
  arXiv  Detail & Related papers  (2020-04-29T16:45:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.