RIVA: Leveraging LLM Agents for Reliable Configuration Drift Detection

• URL: http://arxiv.org/abs/2603.02345v1
• Date: Mon, 02 Mar 2026 19:28:27 GMT
• Title: RIVA: Leveraging LLM Agents for Reliable Configuration Drift Detection
• Abstract summary: Existing agentic systems implicitly assume that the tools they invoke always return correct outputs, making them vulnerable to erroneous tool responses.<n>We introduce RIVA, a novel multi-agent system that performs robust IaC verification even when tools produce incorrect or misleading outputs.<n>Our results show that cross-validation of diverse tool calls enables more reliable autonomous infrastructure verification in production cloud environments.
• Score: 3.494935876363005
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Infrastructure as code (IaC) tools automate cloud provisioning but verifying that deployed systems remain consistent with the IaC specifications remains challenging. Such configuration drift occurs because of bugs in the IaC specification, manual changes, or system updates. Large language model (LLM)-based agentic AI systems can automate the analysis of large volumes of telemetry data, making them suitable for the detection of configuration drift. However, existing agentic systems implicitly assume that the tools they invoke always return correct outputs, making them vulnerable to erroneous tool responses. Since agents cannot distinguish whether an anomalous tool output reflects a real infrastructure problem or a broken tool, such errors may cause missed drift or false alarms, reducing reliability precisely when it is most needed. We introduce RIVA (Robust Infrastructure by Verification Agents), a novel multi-agent system that performs robust IaC verification even when tools produce incorrect or misleading outputs. RIVA employs two specialized agents, a verifier agent and a tool generation agent, that collaborate through iterative cross-validation, multi-perspective verification, and tool call history tracking. Evaluation on the AIOpsLab benchmark demonstrates that RIVA, in the presence of erroneous tool responses, recovers task accuracy from 27.3% when using a baseline ReAct agent to 50.0% on average. RIVA also improves task accuracy 28% to 43.8% without erroneous tool responses. Our results show that cross-validation of diverse tool calls enables more reliable autonomous infrastructure verification in production cloud environments.

Related papers

• Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents [54.050506620820784]
  Agentic Tool Unlearning (ATU) aims to reduce both parametric recall and tool-mediated recovery while preserving normal tool use for retained knowledge.<n>ATU achieves a better balance between target forgetting and retained utility, making unlearning more robust under tool-augmented agent deployment.
  arXiv  Detail & Related papers  (2026-08-21T18:34:48Z)
• Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures [0.0]
  Real-world systems exhibit non-atomic behaviors such as timeouts after dispatch, delayed visibility, and partial state updates.<n>These mismatches lead to reliability issues including duplicate actions, task success, and unnecessary tool executions.<n>A lightweight, verification-aware tool wrapper is introduced that augments tool calls with postcondition verification, verify-before-retry logic, and idempotency keys.
  arXiv  Detail & Related papers  (2026-07-31T16:16:14Z)
• Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems [19.269603229704355]
  Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request.<n>We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools.<n>We deploy the approach in a Fulfillment Center alarm-triage system, where an agent diagnoses alarms against a 44-node SOP over heterogeneous metric backends.
  arXiv  Detail & Related papers  (2026-07-09T00:27:13Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning [82.89535601592739]
  We propose a two-stage pipeline that generates reliable base tool-use trajectories through self-evolving synthesis with multi-level validation.<n>These augmentations introduce distractor tools, indirect or ambiguous user queries, and noisy, multi-format, or erroneous tool outputs.<n>This design enables automatic reward computation via reference matching for standard cases and lightweight judge-assisted verification for special behaviors such as error detection.
  arXiv  Detail & Related papers  (2026-04-10T18:38:52Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• AgentDropoutV2: Optimizing Information Flow in Multi-Agent Systems via Test-Time Rectify-or-Reject Pruning [34.06688334066569]
  AgentDropoutV2 is a test-time rectify-or-reject pruning framework designed to dynamically optimize MAS information flow without retraining.<n>Our approach acts as an active firewall, intercepting agent outputs and employing a retrieval-augmented to iteratively correct errors.<n> Empirical results on extensive math benchmarks show that AgentDropoutV2 significantly boosts the MAS's task performance.
  arXiv  Detail & Related papers  (2026-02-26T17:31:43Z)
• Veri-Sure: A Contract-Aware Multi-Agent Framework with Temporal Tracing and Formal Verification for Correct RTL Code Generation [4.723302382132762]
  silicon-grade correctness remains bottlenecked by: (i) limited test coverage and reliability of simulation-centric evaluation, (ii) regressions and repair hallucinations, and (iii) semantic drift as intent is reinterpreted across agent handoffs.<n>We propose Veri-Sure, a multi-agent framework that establishes a design contract to align agents' intent and uses a patching mechanism guided by static dependency slicing to perform precise, localized repairs.
  arXiv  Detail & Related papers  (2026-01-27T16:10:23Z)
• The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check [54.08619694620588]
  We present a comprehensive evaluation of dLLMs across two distinct agentic paradigms: Embodied Agents and Tool-Calling Agents.<n>Our results on Agentboard and BFCL reveal a "bitter lesson": current dLLMs fail to serve as reliable agentic backbones.
  arXiv  Detail & Related papers  (2026-01-19T11:45:39Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• Internal Representations as Indicators of Hallucinations in Agent Tool Selection [5.2107604548805915]
  Large Language Models (LLMs) have shown remarkable capabilities in tool calling and tool usage.<n>LLMs suffer from hallucinations where they choose incorrect tools, provide malformed parameters and exhibit 'tool bypass' behavior.<n>We present a computationally efficient framework that detects tool-calling hallucinations in real-time by leveraging LLMs' internal representations.
  arXiv  Detail & Related papers  (2026-01-08T18:38:45Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Multi-Agent Systems for Dataset Adaptation in Software Engineering: Capabilities, Limitations, and Future Directions [8.97512410819274]
  This paper presents the first empirical study on how state-of-the-art multi-agent systems perform in dataset adaptation tasks.<n>We evaluate GitHub Copilot on adapting SE research artifacts from benchmark repositories including ROCODE and LogHub2.0.<n>Results show that current systems can identify key files and generate partial adaptations but rarely produce correct implementations.
  arXiv  Detail & Related papers  (2025-11-26T13:26:11Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on $τ$-bench [58.114899897566964]
  In a multi-turn conversational environment, large language models (LLMs) often struggle with consistent reasoning and adherence to domain-specific policies.<n>We propose the Input-Reformulation Multi-Agent (IRMA) framework, which automatically reformulates user queries augmented with relevant domain rules.<n>IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively.
  arXiv  Detail & Related papers  (2025-08-28T15:57:33Z)
• T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Search [51.91311158085973]
  multimodal misinformation often arises from mixed forgery sources, requiring dynamic reasoning and adaptive verification.<n>We propose T2Agent, a novel misinformation detection agent that incorporates a toolkit with Monte Carlo Tree Search.<n>Extensive experiments show that T2Agent consistently outperforms existing baselines on challenging mixed-source multimodal misinformation benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T09:50:55Z)
• REDO: Execution-Free Runtime Error Detection for COding Agents [3.9903610503301072]
  Execution-free Error Detection for COding Agents (REDO) is a method that integrates runtime errors with static analysis tools. We demonstrate that REDO outperforms current state-of-the-art methods by achieving a 11.0% higher accuracy and a 9.1% higher weighted F1 score.
  arXiv  Detail & Related papers  (2024-10-10T18:06:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.