LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment

• URL: http://arxiv.org/abs/2608.28490v1
• Date: Fri, 28 Aug 2026 16:19:01 GMT
• Title: LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
• Abstract summary: Large language model (LLM)-based agents can plan, use tools, retain state, and revise actions across multi-step.<n>Yet to this date, there remains a lack of systematic understanding of what has been done and how far we are in this field.<n>This paper provides a systematic literature review of the technical approaches, including agent architecture, perception, memory, reasoning and planning, action space, orchestration, and self-improvement.
• Score: 13.382767307824873
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software and systems security workflows are typically procedural: analysts inspect heterogeneous artifacts, form hypotheses, invoke tools, interpret outputs, and revise plans. Large language model (LLM)-based agents, which can plan, use tools, retain state, and revise actions across multi-step workflows, are being rapidly adopted to automate this work. Given the consequences of delegating security decisions to autonomous systems, understanding how such agents are built, used, and assessed is crucial. Yet to this date, there remains a lack of systematic understanding of what has been done and how far we are in this field: the term "agent" is applied inconsistently, applications differ sharply in risk, and assessment protocols are often incomparable. To gain a comprehensive and coherent view of this area hence inform relevant future research, this paper provides a systematic literature review of the (1) technical approaches, including agent architecture, perception, memory, reasoning and planning, action space, orchestration, and self-improvement, (2) applications, with respect to the security tasks served, and (3) assessment, including the datasets, outcome and trajectory metrics, safety measures, and baselines considered, over the peer-reviewed literature spanning the emergence of this area (2023--2026). Our synthesis reveals a field that has built agents able to act but not yet agents whose authority is bounded or whose behavior is auditable. In addition to knowledge systematization, we also extend our insights into the limitations of and challenges faced by current approach, application, and assessment designs, which shed light on potentially promising future research directions.

Related papers

• Engineering Trustworthy Agentic AI for Critical Systems [1.8015174793525714]
  The study adopts a trustworthiness model organized around five cross-cutting dimensions: safety and constraint satisfaction; robustness and reliability; transparency and interpretability; accountability and auditability; and privacy and security.<n>Building on this foundation, agentic systems architectures, threats, concrete trust mechanisms, and quantitative metrics are surveyed for direct application in agentic systems development and evaluation.<n>These principles are then examined across four constraint-bound engineering domains: power systems, autonomous vehicles/robotics/UAVs, high-performance computing, and communication networks.
  arXiv  Detail & Related papers  (2026-07-20T22:28:16Z)
• Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems [64.88044474617854]
  The rapid integration of large language model-based agents into recommender systems has driven a shift from static, ranking-based pipelines toward autonomous and interactive systems.<n>This survey introduces a unified taxonomy grounded in the level of autonomy and three core paradigms of agentic recommender systems.
  arXiv  Detail & Related papers  (2026-07-05T17:55:05Z)
• From Question Answering to Task Completion: A Survey on Agent System and Harness Design [67.8241530947325]
  This survey examines LLM-based agents through a model-harness lens.<n>We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model and an execution harness.<n>We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities.
  arXiv  Detail & Related papers  (2026-06-14T05:40:16Z)
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security [57.35851886874902]
  Agentic AI systems execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness.<n>This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments.<n>For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies.
  arXiv  Detail & Related papers  (2026-05-17T10:26:37Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• AI Agent Systems: Architectures, Applications, and Evaluation [4.967019713320407]
  AI agents combine foundation models with reasoning, planning, memory, and tool use.<n>We organize prior work into a unified taxonomy spanning agent components.<n>We discuss key design trade-offs -- latency vs. accuracy, autonomy vs. controllability, and capability vs. reliability.
  arXiv  Detail & Related papers  (2026-01-05T02:38:40Z)
• Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems [0.0]
  Recent advances in agentic AI have shifted the focus from standalone Large Language Models to integrated systems.<n>We propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment.<n>We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations by conventional metrics.
  arXiv  Detail & Related papers  (2025-12-14T18:17:40Z)
• Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents [63.03252293761656]
  This paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on large language models (LLMs)<n>We examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use.<n>We provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation.
  arXiv  Detail & Related papers  (2025-10-20T12:46:55Z)
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System [56.40989626804489]
  This survey provides the first holistic analysis of Large Language Models-powered software engineering.<n>We review over 150 recent papers and propose a taxonomy along two key dimensions: (1) Solutions, categorized into prompt-based, fine-tuning-based, and agent-based paradigms, and (2) Benchmarks, including tasks such as code generation, translation, and repair.
  arXiv  Detail & Related papers  (2025-10-10T06:56:50Z)
• Evaluation and Benchmarking of LLM Agents: A Survey [2.75311233296471]
  This survey introduces a two-dimensional taxonomy that organizes existing work along evaluation objectives.<n>We highlight enterprise-specific challenges, such as role-based access to data.<n>We also identify future research directions, including holistic, more realistic, and scalable evaluation.
  arXiv  Detail & Related papers  (2025-07-29T04:57:02Z)
• Survey on Evaluation of LLM-based Agents [28.91672694491855]
  The emergence of LLM-based agents represents a paradigm shift in AI.<n>This paper provides the first comprehensive survey of evaluation methodologies for these increasingly capable agents.
  arXiv  Detail & Related papers  (2025-03-20T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.