LLMCloudHunter: Harnessing LLMs for Automated Extraction of Detection Rules from Cloud-Based CTI

• URL: http://arxiv.org/abs/2407.05194v1
• Date: Sat, 6 Jul 2024 21:43:35 GMT
• Title: LLMCloudHunter: Harnessing LLMs for Automated Extraction of Detection Rules from Cloud-Based CTI
• Authors: Yuval Schwartz, Lavi Benshimol, Dudu Mimran, Yuval Elovici, Asaf Shabtai,
• Abstract summary: Open-source cyber threat intelligence (OS-CTI) is a valuable resource for threat hunters. Previous studies aimed at automating OSCTI analysis failed to provide actionable outputs. We propose LLMCloudHunter, a novel framework that automatically generates generic-signature detection rule candidates from OSCTI data.
• Score: 24.312198733476063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the number and sophistication of cyber attacks have increased, threat hunting has become a critical aspect of active security, enabling proactive detection and mitigation of threats before they cause significant harm. Open-source cyber threat intelligence (OS-CTI) is a valuable resource for threat hunters, however, it often comes in unstructured formats that require further manual analysis. Previous studies aimed at automating OSCTI analysis are limited since (1) they failed to provide actionable outputs, (2) they did not take advantage of images present in OSCTI sources, and (3) they focused on on-premises environments, overlooking the growing importance of cloud environments. To address these gaps, we propose LLMCloudHunter, a novel framework that leverages large language models (LLMs) to automatically generate generic-signature detection rule candidates from textual and visual OSCTI data. We evaluated the quality of the rules generated by the proposed framework using 12 annotated real-world cloud threat reports. The results show that our framework achieved a precision of 92% and recall of 98% for the task of accurately extracting API calls made by the threat actor and a precision of 99% with a recall of 98% for IoCs. Additionally, 99.18% of the generated detection rule candidates were successfully compiled and converted into Splunk queries.

Related papers

• Benchmarking Large Language Models for Zero-shot and Few-shot Phishing URL Detection [0.0]
  Deceptive URLs have reached unprecedented sophistication due to the widespread use of generative AI by cybercriminals.<n> phishing volume has escalated over 4,000% since 2022, with nearly 50% more attacks evading detection.<n>We present a benchmark of LLMs under a unified zero-shot and few-shot prompting framework.
  arXiv  Detail & Related papers  (2026-02-02T18:56:06Z)
• Proactively Detecting Threats: A Novel Approach Using LLMs [0.0]
  This paper presents the first systematic evaluation of large language models (LLMs) to proactively identify indicators of compromise (IOCs)<n>We developed an automated system that pulls IOCs from 15 web-based threat report sources to evaluate six LLM models.<n>Gemini 1.5 Pro achieved 0.958 precision and 0.788 specificity for malicious IOC identification, while demonstrating perfect recall (1.0) for actual threats.
  arXiv  Detail & Related papers  (2026-01-13T23:28:33Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• External Data Extraction Attacks against Retrieval-Augmented Large Language Models [70.47869786522782]
  RAG has emerged as a key paradigm for enhancing large language models (LLMs)<n>RAG introduces new risks of external data extraction attacks (EDEAs), where sensitive or copyrighted data in its knowledge base may be extracted verbatim.<n>We present the first comprehensive study to formalize EDEAs against retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2025-10-03T12:53:45Z)
• Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents [4.303444472156151]
  Large Language Model (LLM)-enabled agents are rapidly emerging across a wide range of applications.<n>This work presents the first study of time-of-check to time-of-use (TOCTOU) vulnerabilities in LLM-enabled agents.<n>We introduce TOCTOU-Bench, a benchmark with 66 realistic user tasks designed to evaluate this class of vulnerabilities.
  arXiv  Detail & Related papers  (2025-08-23T22:41:49Z)
• CyberGym: Evaluating AI Agents' Cybersecurity Capabilities with Real-World Vulnerabilities at Scale [46.76144797837242]
  Large language model (LLM) agents are becoming increasingly skilled at handling cybersecurity tasks autonomously.<n>Existing benchmarks fall short, often failing to capture real-world scenarios or being limited in scope.<n>We introduce CyberGym, a large-scale and high-quality cybersecurity evaluation framework featuring 1,507 real-world vulnerabilities.
  arXiv  Detail & Related papers  (2025-06-03T07:35:14Z)
• LLM-Assisted Proactive Threat Intelligence for Automated Reasoning [2.0427650128177]
  This research presents a novel approach to enhance real-time cybersecurity threat detection and response. We integrate large language models (LLMs) and Retrieval-Augmented Generation (RAG) systems with continuous threat intelligence feeds.
  arXiv  Detail & Related papers  (2025-04-01T05:19:33Z)
• Knowledge Transfer from LLMs to Provenance Analysis: A Semantic-Augmented Method for APT Detection [1.2571354974258824]
  We propose a new strategy for taking advantage of Large Language Models (LLMs) in provenance-based threat detection. LLMs offer additional details in provenance data interpretation, leveraging their knowledge of system calls, software identity, and high-level understanding of application execution context. In our evaluation, supervised threat detection achieves a precision of 99.0%, and semi-supervised anomaly detection attains a precision of 96.9%.
  arXiv  Detail & Related papers  (2025-03-24T03:51:09Z)
• TIPS: Threat Actor Informed Prioritization of Applications using SecEncoder [10.80485109546937]
  TIPS combines the strengths of both encoder and decoder language models to detect and prioritize compromised applications. In real-world scenarios, TIPS significantly reduces the backlog of investigations for security analysts by 87%.
  arXiv  Detail & Related papers  (2024-11-12T03:33:08Z)
• Attention Tracker: Detecting Prompt Injection Attacks in LLMs [62.247841717696765]
  Large Language Models (LLMs) have revolutionized various domains but remain vulnerable to prompt injection attacks. We introduce the concept of the distraction effect, where specific attention heads shift focus from the original instruction to the injected instruction. We propose Attention Tracker, a training-free detection method that tracks attention patterns on instruction to detect prompt injection attacks.
  arXiv  Detail & Related papers  (2024-11-01T04:05:59Z)
• ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMs [1.7191671053507043]
  Security analysts face the challenge of mitigating newly discovered vulnerabilities in real-time. Over 300,000 Common Vulnerabilities and Exposures have been identified since 1999. Over 25,000 vulnerabilities have been identified so far in 2024.
  arXiv  Detail & Related papers  (2024-10-22T20:28:57Z)
• Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy [65.80757820884476]
  We expose a critical yet underexplored vulnerability in the deployment of unlearning systems. We present a threat model where an attacker can degrade model accuracy by submitting adversarial unlearning requests for data not present in the training set. We evaluate various verification mechanisms to detect the legitimacy of unlearning requests and reveal the challenges in verification.
  arXiv  Detail & Related papers  (2024-10-12T16:47:04Z)
• KGV: Integrating Large Language Models with Knowledge Graphs for Cyber Threat Intelligence Credibility Assessment [38.312774244521]
  We propose a knowledge graph-based verifier for Cyber Threat Intelligence (CTI) quality assessment framework. Our approach introduces Large Language Models (LLMs) to automatically extract OSCTI key claims to be verified. To fill the gap in the research field, we created and made public the first dataset for threat intelligence assessment from heterogeneous sources.
  arXiv  Detail & Related papers  (2024-08-15T11:32:46Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• On the Security Risks of Knowledge Graph Reasoning [71.64027889145261]
  We systematize the security threats to KGR according to the adversary's objectives, knowledge, and attack vectors. We present ROAR, a new class of attacks that instantiate a variety of such threats. We explore potential countermeasures against ROAR, including filtering of potentially poisoning knowledge and training with adversarially augmented queries.
  arXiv  Detail & Related papers  (2023-05-03T18:47:42Z)
• Unsupervised User-Based Insider Threat Detection Using Bayesian Gaussian Mixture Models [0.0]
  In this paper, we propose an unsupervised insider threat detection system based on audit data. The proposed approach leverages a user-based model to optimize specific behaviors modelization and an automatic feature extraction system based on Word2Vec. Results indicate that the proposed method competes with state-of-the-art approaches, presenting a good recall of 88%, accuracy and true negative rate of 93%, and a false positive rate of 6.9%.
  arXiv  Detail & Related papers  (2022-11-23T13:45:19Z)
• A System for Efficiently Hunting for Cyber Threats in Computer Systems Using Threat Intelligence [78.23170229258162]
  We build ThreatRaptor, a system that facilitates cyber threat hunting in computer systems using OSCTI. ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, and (3) a query synthesis mechanism that automatically synthesizes a TBQL query from the extracted threat behaviors.
  arXiv  Detail & Related papers  (2021-01-17T19:44:09Z)
• Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence [94.94833077653998]
  ThreatRaptor is a system that facilitates threat hunting in computer systems using open-source Cyber Threat Intelligence (OSCTI) It extracts structured threat behaviors from unstructured OSCTI text and uses a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities. Evaluations on a broad set of attack cases demonstrate the accuracy and efficiency of ThreatRaptor in practical threat hunting.
  arXiv  Detail & Related papers  (2020-10-26T14:54:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.