Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Models

• URL: http://arxiv.org/abs/2601.03287v1
• Date: Sun, 04 Jan 2026 01:39:20 GMT
• Title: Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Models
• Abstract summary: This paper investigates whether Large Language Models (LLMs) can augment post-incident review by autonomously analysing system evidence and identifying security policy gaps.<n>We present a threat-informed, agentic framework that ingests log data, maps observed behaviours to the MITRE ATT&CK framework, and evaluates organisational security policies for adequacy and compliance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cybersecurity post-incident reviews are essential for identifying control failures and improving organisational resilience, yet they remain labour-intensive, time-consuming, and heavily reliant on expert judgment. This paper investigates whether Large Language Models (LLMs) can augment post-incident review workflows by autonomously analysing system evidence and identifying security policy gaps. We present a threat-informed, agentic framework that ingests log data, maps observed behaviours to the MITRE ATT&CK framework, and evaluates organisational security policies for adequacy and compliance. Using a simulated brute-force attack scenario against a Windows OpenSSH service (MITRE ATT&CK T1110), the system leverages GPT-4o for reasoning, LangGraph for multi-agent workflow orchestration, and LlamaIndex for traceable policy retrieval. Experimental results indicate that the LLM-based pipeline can interpret log-derived evidence, identify insufficient or missing policy controls, and generate actionable remediation recommendations with explicit evidence-to-policy traceability. Unlike prior work that treats log analysis and policy validation as isolated tasks, this study integrates both into a unified end-to-end proof-of-concept post-incident review framework. The findings suggest that LLM-assisted analysis has the potential to improve the efficiency, consistency, and auditability of post-incident evaluations, while highlighting the continued need for human oversight in high-stakes cybersecurity decision-making.

Related papers

• CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents [66.9575676266637]
  Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments.<n>In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution.<n>We develop CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization.
  arXiv  Detail & Related papers  (2026-08-31T01:59:51Z)
• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability [65.56750741053266]
  Large language models (LLMs) have transformed misinformation from a content-centric problem into a broader ecosystem-level security challenge.<n>In this paper, we introduce a role-layer framework to unify these risks and defenses.<n>The role characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification.
  arXiv  Detail & Related papers  (2026-07-11T17:04:17Z)
• AgentFixer: From Failure Detection to Fix Recommendations in LLM Agentic Systems [7.429835301272413]
  The framework includes fifteen failure-detection tools and two root-cause analysis modules.<n>It integrates lightweight rule-based checks with LLM-as-a-judge assessments to support structured incident detection, classification, and repair.<n>We applied the framework to IBM CUGA, evaluating its performance on the AppWorld and WebArena benchmarks.
  arXiv  Detail & Related papers  (2026-02-18T14:55:35Z)
• Detecting Object Tracking Failure via Sequential Hypothesis Testing [80.7891291021747]
  Real-time online object tracking in videos constitutes a core task in computer vision.<n>We propose interpreting object tracking as a sequential hypothesis test, wherein evidence for or against tracking failures is gradually accumulated over time.<n>We propose both supervised and unsupervised variants by leveraging either ground-truth or solely internal tracking information.
  arXiv  Detail & Related papers  (2026-02-13T14:57:15Z)
• Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoning [52.144281362465996]
  We propose EAPO (Evidence-Augmented Policy Optimization) to apply Reinforcement Learning to long-context scenarios.<n>We first establish the Evidence-Augmented Reasoning paradigm, validating via Tree-Structured Evidence Sampling.<n>We then introduce a specialized RL algorithm where a reward model computes a Group-Relative Evidence Reward.<n>To sustain accurate supervision throughout training, we further incorporate an Adaptive Reward-Policy Co-Evolution mechanism.
  arXiv  Detail & Related papers  (2026-01-15T11:40:57Z)
• Agentic AI for Autonomous Defense in Software Supply Chain Security: Beyond Provenance to Vulnerability Mitigation [0.0]
  The current paper includes an example of agentic artificial intelligence (AI) based on autonomous software supply chain security.<n>It combines large language model (LLM)-based reasoning, reinforcement learning (RL), and multi-agent coordination.<n>Results show that agentic AI can facilitate the transition to self defending, proactive software supply chains.
  arXiv  Detail & Related papers  (2025-12-29T14:06:09Z)
• GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision [47.99880677909197]
  GuardTrace-VL is a vision-aware safety auditor that monitors the full Question-Thinking-Answer (QTA) pipeline via joint image-text analysis.<n>We propose a three-stage progressive training scheme combined with the data refinement process, enabling the model to learn nuanced and context-dependent safety preferences.<n>On our proposed test set covering both in-domain and out-of-domain scenarios, GuardTrace-VL model achieves an F1 score of 93.1% on unsafe reasoning detection tasks.
  arXiv  Detail & Related papers  (2025-11-26T02:49:51Z)
• LLM-driven Provenance Forensics for Threat Investigation and Detection [12.388936704058521]
  PROVSEEK is an agentic framework for provenance-driven forensic analysis and threat intelligence extraction.<n>It generates precise, context-aware queries that fuse a vectorized threat report knowledge base with data from system provenance databases.<n>It resolves provenance queries, orchestrates multiple role-specific agents to mitigate hallucinations, and synthesizes structured, ground-truth verifiable forensic summaries.
  arXiv  Detail & Related papers  (2025-08-29T04:39:52Z)
• A Dynamical Systems Framework for Reinforcement Learning Safety and Robustness Verification [1.104960878651584]
  This paper introduces a novel framework that addresses the lack of formal methods for verifying the robustness and safety of learned policies.<n>By leveraging tools from dynamical systems theory, we identify and visualize Lagrangian Coherent Structures (LCS) that act as the hidden "skeleton" governing the system's behavior.<n>We show that this framework provides a comprehensive and interpretable assessment of policy behavior, successfully identifying critical flaws in policies that appear successful based on reward alone.
  arXiv  Detail & Related papers  (2025-08-21T14:00:26Z)
• Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimization [45.799380822683034]
  We present an extensive study aimed at advancing RL-based finetuning techniques for Large Language Models (LLMs)<n>We highlight key limitations of commonly adopted LLMs, such as their tendency to over-predict certain types of vulnerabilities while failing to detect others.<n>To address this challenge, we explore the use of Group Relative Policy Optimization (GRPO), a recent policy-gradient method, for guiding LLM behavior through structured, rule-based rewards.
  arXiv  Detail & Related papers  (2025-07-03T11:52:45Z)
• A Survey on Model Extraction Attacks and Defenses for Large Language Models [55.60375624503877]
  Model extraction attacks pose significant security threats to deployed language models.<n>This survey provides a comprehensive taxonomy of extraction attacks and defenses, categorizing attacks into functionality extraction, training data extraction, and prompt-targeted attacks.<n>We examine defense mechanisms organized into model protection, data privacy protection, and prompt-targeted strategies, evaluating their effectiveness across different deployment scenarios.
  arXiv  Detail & Related papers  (2025-06-26T22:02:01Z)
• ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis [14.24781559851732]
  We present ETrace, a novel event-driven vulnerability detection framework for smart contracts.<n>By extracting fine-grained event sequences from transaction logs, the framework leverages Large Language Models (LLMs) as adaptive semantic interpreters.<n>ETrace implements pattern-matching to establish causal links between transaction behavior patterns and known attack behaviors.
  arXiv  Detail & Related papers  (2025-06-18T18:18:19Z)
• LLM-Safety Evaluations Lack Robustness [58.334290876531036]
  We argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of noise.<n>We propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers.
  arXiv  Detail & Related papers  (2025-03-04T12:55:07Z)
• Exploring Answer Set Programming for Provenance Graph-Based Cyber Threat Detection: A Novel Approach [4.302577059401172]
  Provenance graphs are useful tools for representing system-level activities in cybersecurity.<n>This paper presents a novel approach using ASP to model and analyze provenance graphs.
  arXiv  Detail & Related papers  (2025-01-24T14:57:27Z)
• It Is Time To Steer: A Scalable Framework for Analysis-driven Attack Graph Generation [50.06412862964449]
  Attack Graph (AG) represents the best-suited solution to support cyber risk assessment for multi-step attacks on computer networks. Current solutions propose to address the generation problem from the algorithmic perspective and postulate the analysis only after the generation is complete. This paper rethinks the classic AG analysis through a novel workflow in which the analyst can query the system anytime.
  arXiv  Detail & Related papers  (2023-12-27T10:44:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.