Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety

• URL: http://arxiv.org/abs/2605.12729v1
• Date: Tue, 12 May 2026 20:31:41 GMT
• Title: Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety
• Abstract summary: Large language models are increasingly being used to support network operations (NetOps) and artificial intelligence for IT operations (AIOps)<n>In both NetOps and AIOps, this shift is changing how tasks are managed.<n>Agent-based operations work as managed, from gathering evidence to taking action, following permissions, policies, and checks, and providing rollback options when necessary.
• Score: 14.195825134031795
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly being used to support network operations (NetOps) and artificial intelligence for IT operations (AIOps), including incident investigation, root-cause analysis, configuration synthesis, and limited self-healing. In both NetOps and AIOps, this shift is changing how tasks are managed. Agent-based operations work as workflows, from gathering evidence to taking action, following permissions, policies, and checks, and providing rollback options when necessary. This is crucial because operational decisions can have instant impacts. To make the argument concrete, we organise the relevant literature around the hierarchy of autonomy, tool scope, evidence traces, and assurance contracts. These contracts define what an agent may observe, propose, and execute. They also define the checks that must pass before any action is allowed. A consistent pattern appears across work on telemetry query recommendation, diagnosis, root-cause analysis, configuration synthesis, change planning, and limited self-healing. Operational reliability does not come chiefly from the model itself. It depends on the machinery around the model. We also argue that evaluation should go beyond static question answering. Agentic NetOps and AIOps systems require workflow-centred evaluation, including trace quality, bounded tool use, safe proposal generation, replay in sandboxed environments, and canary trials with rollback-aware scoring. Without these measures, a system may appear robust yet remain too fragile. Finally, we examine security, privacy, and governance risks that become acute when agents sit close to operational control surfaces. Taken together, the survey concludes that progress in intelligent NetOps and AIOps will depend on treating autonomy as a constrained operational control problem, whose outputs must be reliable, auditable, and securely deployable.

Related papers

• Formal Verification of Agentic Systems over Operational Data [59.98246281888422]
  Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world where they act on persistent operational data.<n>Existing approaches do not provide such system-level guarantees.<n>We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data.
  arXiv  Detail & Related papers  (2026-08-04T13:01:30Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• From Question Answering to Task Completion: A Survey on Agent System and Harness Design [67.8241530947325]
  This survey examines LLM-based agents through a model-harness lens.<n>We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model and an execution harness.<n>We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities.
  arXiv  Detail & Related papers  (2026-06-14T05:40:16Z)
• The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? [80.24951682268332]
  We introduce the Meta-Agent Challenge (MAC), an evaluation framework designed to test the capacity of frontier models for autonomous agent development.<n>To ensure evaluation integrity, this framework is secured by multi-layer defenses against reward hacking.<n>We demonstrate that meta-agents rarely match human-engineered baseline policies, and the few that do are dominated by proprietary frontier models.
  arXiv  Detail & Related papers  (2026-06-03T04:58:17Z)
• Agent Operating Systems (AOS): Integrating Agentic Control Planes into, and Beyond, Traditional Operating Systems [0.04756202129182093]
  This paper introduces the concept of an Agent Operating System (AOS), a systems architecture that integrates an agentic control plane into existing operating systems.<n>We provide a precise definition of an AOS, explicit assumptions and non-goals, and a structured decomposition of AOS responsibilities into schedulers, context and memory management, tool and capability registries, policy and trust enforcement, and observability and audit.<n>We present security and safety implications, including agent specific threat models, and define evaluation criteria that emphasize deterministic enforcement, auditability, and operator comprehensibility.
  arXiv  Detail & Related papers  (2026-06-01T00:08:03Z)
• Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability [21.333731425525205]
  The article develops an architecture-lifecycle framework for deployment-grounded reliability in CUAs.<n>The analysis synthesizes representative systems,benchmarks, and security/privacy studies.
  arXiv  Detail & Related papers  (2026-05-08T01:38:46Z)
• Beyond the Black Box: Interpretability of Agentic AI Tool Use [0.0]
  We introduce a mechanistic-interpretability toolkit built on Sparse Autoencoders and linear probes.<n>The framework reads model states before each action and infers both whether a tool is needed and how consequential the next tool action is likely to be.<n>We train the probes on multi-step trajectories from the NVIDIA Nemotron function-calling dataset and apply the same workflow to GPT-OSS 20B and Gemma 3 27B models.
  arXiv  Detail & Related papers  (2026-05-07T19:47:30Z)
• Clawed and Dangerous: Can We Trust Open Agentic Systems? [12.04979073308511]
  Open agentic systems combine planning with external capabilities, persistent memory, and privileged execution.<n>This paper introduces a six-dimensional analytical taxonomy and synthesizes 50 papers spanning attacks, benchmarks, defenses, audits, and adjacent engineering foundations.<n>Our review shows that the literature is relatively mature in attack characterization and benchmark construction, but remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation.
  arXiv  Detail & Related papers  (2026-03-27T09:45:12Z)
• A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governance [0.22940141855172028]
  This paper presents an assurance framework for Agentic AI systems using Large Language Models (LLMs)<n>Executions are instrumented as Message-Action Traces (MAT) with explicit step and trace contracts.<n>The framework includes stress testing, formulated as a budgeted counterexample search over bounded perturbations.
  arXiv  Detail & Related papers  (2026-03-18T10:23:48Z)
• When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making [68.12864562049957]
  Embodied robotic systems increasingly rely on large language model (LLM)-based agents to support high-level reasoning.<n>This raises a fundamental question for embodied agents: when should the agent reason, and when should it act?<n>We propose RARRL (Resource-Aware Reasoning via Reinforcement Learning), a hierarchical framework for resource-aware orchestration of embodied agents.
  arXiv  Detail & Related papers  (2026-03-17T15:38:50Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents [60.98294016925157]
  AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss.<n>We introduce Single-Shot Planning for CUAs, where a trusted planner generates a complete execution graph with conditional branches before any observation of potentially malicious content.<n>Although this architectural isolation successfully prevents instruction injections, we show that additional measures are needed to prevent Branch Steering attacks.
  arXiv  Detail & Related papers  (2026-01-14T23:06:35Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.