From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness

• URL: http://arxiv.org/abs/2608.00672v1
• Date: Sat, 01 Aug 2026 13:47:44 GMT
• Title: From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness
• Abstract summary: Security Operations Centers (SOCs) process large volumes of security events.<n>Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations.<n>Despite concerns, practitioners express a strong willingness to adopt LLMs.
• Score: 9.803413284017138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Security Operations Centers (SOCs) process large volumes of security events, requiring analysts to accurately detect and assess ongoing cyberattacks under time pressure. Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations, yet their practical suitability for real-world SOC workflows remains poorly understood. To address this gap, we conducted 25 semi-structured interviews with SOC practitioners who had prior experience with LLMs, complemented by interactive scenarios to anticipate challenges and identify opportunities for the responsible integration of LLM-based tools into SOC workflows. We identified 15 LLM use cases grouped into six functional categories. While LLMs are valued for automating repetitive, low-level tasks such as report automation, practitioners rate high-impact tasks such as incident analysis as not yet feasible, reporting limitations in technical depth, context awareness, and organization-specific knowledge. They locate these limitations less in the models than in the readiness of their SOCs and human factors driving over-reliance. Despite concerns, practitioners express a strong willingness to adopt LLMs, describing competitive pressure that leaves few alternatives. This work contributes an empirical, practitioner-driven analysis of LLM use across SOC roles and organizations and derives concrete design and integration requirements for human-centered, operationally safe LLM-assisted security operations.

Related papers

• Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers [3.4345394794323276]
  Large Language Models (LLMs) are increasingly being explored within Security Operation Centers (SOCs) to support text-heavy analytical work.<n>Despite this interest, practitioners describe critical operational concerns, most notably hallucinations and opaque reasoning.<n>We present findings from semi-structured interviews with 20 SOC practitioners spanning frontline analysts, SOC managers, and tool developers.
  arXiv  Detail & Related papers  (2026-08-17T21:37:33Z)
• Hybrid Analysis for Secure MCP Tool Use in LLM Agents [77.68135853475535]
  We propose a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in large language model (LLM) agents.<n> MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
  arXiv  Detail & Related papers  (2026-07-28T05:17:34Z)
• Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability [65.56750741053266]
  Large language models (LLMs) have transformed misinformation from a content-centric problem into a broader ecosystem-level security challenge.<n>In this paper, we introduce a role-layer framework to unify these risks and defenses.<n>The role characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification.
  arXiv  Detail & Related papers  (2026-07-11T17:04:17Z)
• Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit [3.7217825105232456]
  Large language models (LLMs) have emerged as promising tools for augmenting Security Operations Center (SOC)<n>We analyzed 892 posts between December 2022 and September 2025 from three cybersecurity-focused forums on Reddit.<n>Our findings reveal nuanced patterns in LLM tools adoption, highlighting independent use of LLMs for low-risk, productivity-oriented tasks.
  arXiv  Detail & Related papers  (2026-04-11T03:00:37Z)
• CTIArena: Benchmarking LLM Knowledge and Reasoning Across Heterogeneous Cyber Threat Intelligence [48.63397742510097]
  Cyber threat intelligence (CTI) is central to modern cybersecurity, providing critical insights for detecting and mitigating evolving threats.<n>With the natural language understanding and reasoning capabilities of large language models (LLMs), there is increasing interest in applying them to CTI.<n>We present CTIArena, the first benchmark for evaluating LLM performance on heterogeneous, multi-source CTI.
  arXiv  Detail & Related papers  (2025-10-13T22:10:17Z)
• Large Language Models for Security Operations Centers: A Comprehensive Survey [0.0]
  Large Language Models (LLMs) have emerged as powerful tools capable of understanding and generating human-like text.<n>This survey systematically explores the integration of generative AI and more specifically LLMs into SOC workflow.
  arXiv  Detail & Related papers  (2025-09-13T15:27:50Z)
• LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres [15.150052598613408]
  The integration of Large Language Models into Security Operations Centres (SOCs) presents a transformative, yet still evolving, opportunity to reduce analyst workload.<n>We present a longitudinal study of 3,090 analyst queries from 45 SOC analysts over 10 months.<n>Our analysis reveals that analysts use LLMs as on-demand aids for sensemaking and context-building, rather than for making high-stakes determinations.
  arXiv  Detail & Related papers  (2025-08-26T11:40:02Z)
• AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents [48.925168866726814]
  AgentAuditor is a universal, training-free, memory-augmented reasoning framework.<n>ASSEBench is the first benchmark designed to check how well LLM-based evaluators can spot both safety risks and security threats.
  arXiv  Detail & Related papers  (2025-05-31T17:10:23Z)
• Benchmarking LLMs in an Embodied Environment for Blue Team Threat Hlunting [14.810934670172479]
  Large Language Models (LLMs) offer promising capabilities for enhancing threat analysis.<n>Their effectiveness in real-world blue team threat-hunting scenarios remains insufficiently explored.<n>We present CYBERTEAM, a benchmark designed to guide LLMs in blue teaming practice.
  arXiv  Detail & Related papers  (2025-05-17T08:33:50Z)
• Global Challenge for Safe and Secure LLMs Track 1 [57.08717321907755]
  The Global Challenge for Safe and Secure Large Language Models (LLMs) is a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-11-21T08:20:31Z)
• Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate [74.06294042304415]
  We propose ScaleEval, an agent-debate-assisted meta-evaluation framework. We release the code for our framework, which is publicly available on GitHub.
  arXiv  Detail & Related papers  (2024-01-30T07:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.