Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit

• URL: http://arxiv.org/abs/2604.09998v1
• Date: Sat, 11 Apr 2026 03:00:37 GMT
• Title: Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit
• Abstract summary: Large language models (LLMs) have emerged as promising tools for augmenting Security Operations Center (SOC)<n>We analyzed 892 posts between December 2022 and September 2025 from three cybersecurity-focused forums on Reddit.<n>Our findings reveal nuanced patterns in LLM tools adoption, highlighting independent use of LLMs for low-risk, productivity-oriented tasks.
• Score: 3.7217825105232456
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) have recently emerged as promising tools for augmenting Security Operations Center (SOC) workflows, with vendors increasingly marketing autonomous AI solutions for SOCs. However, there remains a limited empirical understanding of how such tools are used, perceived, and adopted by real-world security practitioners. To address this gap, we conduct a mixed-methods analysis of discussions in cybersecurity-focused forums to learn how a diverse group of practitioners use and perceive modern LLM tools for security operations. More specifically, we analyzed 892 posts between December 2022 and September 2025 from three cybersecurity-focused forums on Reddit, and, using a combination of qualitative coding and statistical analysis, examined how security practitioners discuss LLM tools across three dimensions: (1) their stated tools and use cases, (2) the perceived pros and cons of each tool across a set of critical factors, and (3) their adoption of such tools and the expected impacts on the cybersecurity industry and individual analysts. Overall, our findings reveal nuanced patterns in LLM tools adoption, highlighting independent use of LLMs for low-risk, productivity-oriented tasks, alongside active interest around enterprise-grade, security-focused LLM platforms. Although practitioners report meaningful gains in efficiency and effectiveness in LLM-assisted workflows, persistent issues with reliability, verification overheads, and security risks sharply constrain the autonomy granted to LLM tools. Based on these results, we also provide recommendations for developing and adopting LLM tools to ensure the security of organizations and the safety of cybersecurity practitioners.

Related papers

• From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness [9.803413284017138]
  Security Operations Centers (SOCs) process large volumes of security events.<n>Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations.<n>Despite concerns, practitioners express a strong willingness to adopt LLMs.
  arXiv  Detail & Related papers  (2026-08-01T13:47:44Z)
• Hybrid Analysis for Secure MCP Tool Use in LLM Agents [77.68135853475535]
  We propose a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in large language model (LLM) agents.<n> MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
  arXiv  Detail & Related papers  (2026-07-28T05:17:34Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• Model-Assisted and Human-Guided: Perceptions and Practices of Software Professionals Using LLMs for Coding [2.198430261120653]
  Large Language Models have quickly become a central component of modern software development.<n>This paper presents preliminary findings from a global survey of 131 software practitioners.
  arXiv  Detail & Related papers  (2025-10-10T06:59:56Z)
• SafeToolBench: Pioneering a Prospective Benchmark to Evaluating Tool Utilization Safety in LLMs [35.180946816997164]
  Large Language Models (LLMs) have exhibited great performance in autonomously calling various tools in external environments.<n>This paper focuses on prospective ways to assess the safety of LLM tool utilization, aiming to avoid irreversible harm caused by directly executing tools.<n>We propose SafeToolBench, the first benchmark to comprehensively assess tool utilization security in a prospective manner.<n>We also propose a novel framework, SafeInstructTool, which aims to enhance LLMs' awareness of tool utilization security from three perspectives.
  arXiv  Detail & Related papers  (2025-09-09T01:31:25Z)
• Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity? [51.96049148869987]
  Large Language Models (LLMs) offer new tools and challenges for the security of computer systems.<n>We investigate whether classical game-theoretic frameworks can effectively capture the behaviours of LLM-driven actors and bots.
  arXiv  Detail & Related papers  (2025-08-04T08:57:14Z)
• RRTL: Red Teaming Reasoning Large Language Models in Tool Learning [8.547055998769476]
  The safety of newly emerging reasoning LLMs (RLLMs) in the context of tool learning remains underexplored.<n>We propose RRTL, a red teaming approach specifically designed to evaluate RLLMs in tool learning.<n>We conduct a comprehensive evaluation on seven mainstream RLLMs and uncover three key findings.
  arXiv  Detail & Related papers  (2025-05-21T10:21:19Z)
• Global Challenge for Safe and Secure LLMs Track 1 [57.08717321907755]
  The Global Challenge for Safe and Secure Large Language Models (LLMs) is a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-11-21T08:20:31Z)
• Evaluating the Usability of LLMs in Threat Intelligence Enrichment [0.30723404270319693]
  Large Language Models (LLMs) have the potential to significantly enhance threat intelligence. However, concerns about their reliability, accuracy, and potential for generating inaccurate information persist. This study conducts a comprehensive usability evaluation of five LLMs ChatGPT, Gemini, Cohere, Copilot, and Meta AI.
  arXiv  Detail & Related papers  (2024-09-23T14:44:56Z)
• Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents [56.822238860147024]
  Augmenting large language models with external tools has emerged as a promising approach to extend their utility.<n>Previous methods manually parse tool documentation and create in-context demonstrations, transforming tools into structured formats for LLMs to use in their step-by-step reasoning.<n>We propose AutoTools, a framework that enables LLMs to automate the tool-use workflow.
  arXiv  Detail & Related papers  (2024-05-26T11:40:58Z)
• Efficient Tool Use with Chain-of-Abstraction Reasoning [63.08202389132155]
  Large language models (LLMs) need to ground their reasoning to real-world knowledge.<n>There remains challenges for fine-tuning LLM agents to invoke tools in multi-step reasoning problems.<n>We propose a new method for LLMs to better leverage tools in multi-step reasoning.
  arXiv  Detail & Related papers  (2024-01-30T21:53:30Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
  Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability. We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization. We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
  arXiv  Detail & Related papers  (2023-05-23T17:51:52Z)
• An Assessment of the Usability of Machine Learning Based Tools for the Security Operations Center [1.5391321019692434]
  By 2024 80% of security operation centers (SOCs) will use machine learning (ML) based solutions to enhance their operations. This work presents the results of the first in situ usability assessment of ML-based tools.
  arXiv  Detail & Related papers  (2020-12-16T15:17:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.