LLMs for Zero-Shot Threat Detection via Structured Risk Indicators

• URL: http://arxiv.org/abs/2608.16508v1
• Date: Mon, 17 Aug 2026 12:47:38 GMT
• Title: LLMs for Zero-Shot Threat Detection via Structured Risk Indicators
• Abstract summary: We propose a two-stage large language model (LLM) framework for zero-shot detection of insider threats and advanced persistent threats (APTs) from heterogeneous security logs.<n>The framework models behavioural activity as chronological timelines and incorporates retrieval-augmented generation (RAG) to provide personalised context from each user's historical activity.<n>It is evaluated on two benchmark datasets, CERT r5.2 for insider threat detection and PicoDomain for APT detection.
• Score: 3.888900238305354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a two-stage large language model (LLM) framework for zero-shot detection of insider threats and advanced persistent threats (APTs) from heterogeneous security logs. The framework models user activity as chronological timelines and incorporates retrieval-augmented generation (RAG) to provide personalised behavioural context from each user's historical activity. Rather than performing end-to-end classification directly from raw logs, it first generates structured, interpretable sets of threat-specific risk indicators, which are then classified jointly across temporal sequences to capture attack patterns spanning multiple windows.The framework is evaluated on two benchmark datasets, CERT r5.2 for insider threat detection and PicoDomain for APT detection, using four combinations of two open-weight LLMs under both retrieval and non-retrieval settings. All configurations outperform the previous state-of-the-art LLM-based framework (GABM), with the best configuration improving the F1-score by 11.40 percentage points on CERT r5.2 and 31.50 percentage points on PicoDomain. Results further show that retrieval mainly benefits weaker LLMs by generating more discriminative risk indicators, whereas stronger models achieve comparable performance without retrieved context. The most effective assignment of LLMs to the two stages depends on the dataset. These findings show that the quality of the generated risk indicators is the main driver of zero-shot cyber threat detection performance.

Related papers

• MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents [5.605743575854166]
  MemRiskBench is a 120-episode scripted benchmark with full trace logging and no LLM-as-judge on the pass/fail path.<n>A risk-preserving subset selector preserves risk-type coverage and high-risk detection using trace-grounded deterministic features.<n>All episodes, traces, the scoring implementation, and the selector are released to support reproducible evaluation and risk assessment of deployed LLM agents.
  arXiv  Detail & Related papers  (2026-09-14T03:31:19Z)
• SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models [73.04773649437375]
  Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior.<n>We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale.<n>We also construct SafeAtlas-Bench, a set of 5,000 instances for evaluating five-level predictions and continuous risk scores.
  arXiv  Detail & Related papers  (2026-08-29T07:08:01Z)
• Reasoning-Aware AIGC Detection via Alignment and Reinforcement [55.09684020007737]
  REVEAL is a framework that generates interpretable reasoning chains before classification.<n>It achieves state-of-the-art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-04-21T07:29:55Z)
• TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories [20.868825285848196]
  Safety guardrails are well-benchmarked for natural language responses, but their efficacy remains largely unexplored within multi-step tool-use trajectories.<n>To address this gap, we introduce StructuralSafe-Bench, the first comprehensive benchmark specifically designed to assess mid-trajectory safety.
  arXiv  Detail & Related papers  (2026-04-08T15:46:14Z)
• Synergistic Directed Execution and LLM-Driven Analysis for Zero-Day AI-Generated Malware Detection [0.12891210250935145]
  Weaponization of LLMs for automated malware generation poses an existential threat to conventional detection paradigms.<n>This paper introduces a novel hybrid analysis framework that combines emphconcolic execution with emphdeep-learning-based vulnerability classification.<n>A benchmark comprising 2,500 LLM-synthesized samples achieves 98.7% accuracy on conventional malware and 97.5% accuracy on AI-generated threats.
  arXiv  Detail & Related papers  (2026-03-10T00:25:41Z)
• Hubble: An LLM-Driven Agentic Framework for Safe, Diverse, and Reproducible Alpha Factor Discovery [0.0]
  We present Hubble, an agentic factor mining framework that combines large language models (LLMs) with a domain-specific operator language.<n>On a U.S. equity universe of roughly 500 stocks, our main run evaluates 104 valid candidates across three rounds with zero runtime crashes.<n>We then fix the resulting top-5 factors and validate them on a held-out period from 2025-06-01 to 2026-03-13.
  arXiv  Detail & Related papers  (2026-03-09T05:21:00Z)
• Mind the Gap: Evaluating LLMs for High-Level Malicious Package Detection vs. Fine-Grained Indicator Identification [1.1103813686369686]
  Large Language Models (LLMs) have emerged as a promising tool for automated security tasks.<n>This paper presents a systematic evaluation of 13 LLMs for detecting malicious software packages.
  arXiv  Detail & Related papers  (2026-02-18T09:36:46Z)
• Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive Trigger Learning [89.1856483797116]
  We introduce BEAT, the first framework to inject visual backdoors into MLLM-based embodied agents.<n>Unlike textual triggers, object triggers exhibit wide variation across viewpoints and lighting, making them difficult to implant reliably.<n>BEAT achieves attack success rates up to 80%, while maintaining strong benign task performance.
  arXiv  Detail & Related papers  (2025-10-31T16:50:49Z)
• Automated Skill Decomposition Meets Expert Ontologies: Bridging the Granularity Gap with LLMs [1.2891210250935148]
  This paper investigates automated skill decomposition using Large Language Models (LLMs)<n>Our framework standardizes the pipeline from prompting and generation to normalization and alignment with ontology nodes.<n>To evaluate outputs, we introduce two metrics: a F1-score that uses optimal embedding-based matching to assess content accuracy, and a hierarchy-aware F1-score that credits structurally correct placements to assess granularity.
  arXiv  Detail & Related papers  (2025-10-13T12:03:06Z)
• Backdoor Cleaning without External Guidance in MLLM Fine-tuning [76.82121084745785]
  Believe Your Eyes (BYE) is a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples.<n>It achieves near-zero attack success rates while maintaining clean-task performance.
  arXiv  Detail & Related papers  (2025-05-22T17:11:58Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.