CYGENT: A cybersecurity conversational agent with log summarization powered by GPT-3

• URL: http://arxiv.org/abs/2403.17160v1
• Date: Mon, 25 Mar 2024 20:17:04 GMT
• Title: CYGENT: A cybersecurity conversational agent with log summarization powered by GPT-3
• Authors: Prasasthy Balasubramanian, Justin Seby, Panos Kostakos,
• Abstract summary: CYGENT is a conversational agent framework powered by GPT-3.5 turbo model. It provides cybersecurity information, analyzing and summarizing uploaded log files, detecting specific events, and delivering essential instructions.
• Score: 0.08192907805418582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In response to the escalating cyber-attacks in the modern IT and IoT landscape, we developed CYGENT, a conversational agent framework powered by GPT-3.5 turbo model, designed to aid system administrators in ensuring optimal performance and uninterrupted resource availability. This study focuses on fine-tuning GPT-3 models for cybersecurity tasks, including conversational AI and generative AI tailored specifically for cybersecurity operations. CYGENT assists users by providing cybersecurity information, analyzing and summarizing uploaded log files, detecting specific events, and delivering essential instructions. The conversational agent was developed based on the GPT-3.5 turbo model. We fine-tuned and validated summarizer models (GPT3) using manually generated data points. Using this approach, we achieved a BERTscore of over 97%, indicating GPT-3's enhanced capability in summarizing log files into human-readable formats and providing necessary information to users. Furthermore, we conducted a comparative analysis of GPT-3 models with other Large Language Models (LLMs), including CodeT5-small, CodeT5-base, and CodeT5-base-multi-sum, with the objective of analyzing log analysis techniques. Our analysis consistently demonstrated that Davinci (GPT-3) model outperformed all other LLMs, showcasing higher performance. These findings are crucial for improving human comprehension of logs, particularly in light of the increasing numbers of IoT devices. Additionally, our research suggests that the CodeT5-base-multi-sum model exhibits comparable performance to Davinci to some extent in summarizing logs, indicating its potential as an offline model for this task.

Related papers

• Integrating Large Language Models with Internet of Things Applications [6.22153888560487]
  This paper identifies and analyzes applications in which Large Language Models (LLMs) can make Internet of Things (IoT) networks more intelligent and responsive. Our results reveal that the GPT model under few-shot learning achieves 87.6% detection accuracy, whereas the fine-tuned GPT increases the value to 94.9%.
  arXiv  Detail & Related papers  (2024-10-25T00:21:45Z)
• SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection [0.0]
  Phishing is a major threat to organizations by using social engineering to trick users into revealing sensitive information. In this paper, we investigate whether the remarkable performance of Large Language Models (LLMs) can be leveraged for particular task like text classification. We demonstrate how LLMs can generate convincing phishing emails, making it harder to spot scams.
  arXiv  Detail & Related papers  (2024-06-10T13:13:39Z)
• DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation [83.30006900263744]
  Data analysis is a crucial analytical process to generate in-depth studies and conclusive insights. We propose to automatically generate high-quality answer annotations leveraging the code-generation capabilities of LLMs. Our DACO-RL algorithm is evaluated by human annotators to produce more helpful answers than SFT model in 57.72% cases.
  arXiv  Detail & Related papers  (2024-03-04T22:47:58Z)
• LogShield: A Transformer-based APT Detection System Leveraging Self-Attention [2.1256044139613772]
  This paper proposes LogShield, a framework designed to detect APT attack patterns leveraging the power of self-attention in transformers. We incorporate customized embedding layers to effectively capture the context of event sequences derived from provenance graphs. Our framework achieved superior F1 scores of 98% and 95% on the two datasets respectively, surpassing the F1 scores of 96% and 94% obtained by LSTM models.
  arXiv  Detail & Related papers  (2023-11-09T20:43:15Z)
• Text generation for dataset augmentation in security classification tasks [55.70844429868403]
  This study evaluates the application of natural language text generators to fill this data gap in multiple security-related text classification tasks. We find substantial benefits for GPT-3 data augmentation strategies in situations with severe limitations on known positive-class samples.
  arXiv  Detail & Related papers  (2023-10-22T22:25:14Z)
• SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning [5.177947445379688]
  This study presents a thorough examination of various Generative Pretrained Transformer (GPT) methodologies in sentiment analysis. Three primary strategies are employed: 1) prompt engineering using the advanced GPT-3.5 Turbo, 2) fine-tuning GPT models, and 3) an inventive approach to embedding classification. The research yields detailed comparative insights among these strategies and individual GPT models, revealing their unique strengths and potential limitations.
  arXiv  Detail & Related papers  (2023-07-16T05:33:35Z)
• A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models [71.42197262495056]
  GPT series models have gained considerable attention due to their exceptional natural language processing capabilities. We select six representative models, comprising two GPT-3 series models and four GPT-3.5 series models. We evaluate their performance on nine natural language understanding (NLU) tasks using 21 datasets. Our experiments reveal that the overall ability of GPT series models on NLU tasks does not increase gradually as the models evolve.
  arXiv  Detail & Related papers  (2023-03-18T14:02:04Z)
• Is GPT-3 a Good Data Annotator? [30.9559541574174]
  GPT-3 is a large-scale language model developed by OpenAI. In this paper, we evaluate the performance of GPT-3 as a data annotator.
  arXiv  Detail & Related papers  (2022-12-20T17:28:41Z)
• News Summarization and Evaluation in the Era of GPT-3 [73.48220043216087]
  We study how GPT-3 compares against fine-tuned models trained on large summarization datasets. We show that not only do humans overwhelmingly prefer GPT-3 summaries, prompted using only a task description, but these also do not suffer from common dataset-specific issues such as poor factuality.
  arXiv  Detail & Related papers  (2022-09-26T01:04:52Z)
• A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking [78.2700757742992]
  Task-oriented dialogue systems often employ a Dialogue State Tracker (DST) to successfully complete conversations. Recent state-of-the-art DST implementations rely on schemata of diverse services to improve model robustness. We propose a single multi-task BERT-based model that jointly solves the three DST tasks of intent prediction, requested slot prediction and slot filling.
  arXiv  Detail & Related papers  (2022-07-02T13:27:59Z)
• Graph Backdoor [53.70971502299977]
  We present GTA, the first backdoor attack on graph neural networks (GNNs) GTA departs in significant ways: it defines triggers as specific subgraphs, including both topological structures and descriptive features. It can be instantiated for both transductive (e.g., node classification) and inductive (e.g., graph classification) tasks.
  arXiv  Detail & Related papers  (2020-06-21T19:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.