A Graph-Based Reinforcement Learning Framework for Structured Drift Diagnosis and Recovery in Autonomous LLM Agents

• URL: http://arxiv.org/abs/2608.14109v1
• Date: Fri, 14 Aug 2026 09:12:14 GMT
• Title: A Graph-Based Reinforcement Learning Framework for Structured Drift Diagnosis and Recovery in Autonomous LLM Agents
• Abstract summary: Existing approaches address drift at the prompt level but lack structured mechanisms for step-level detection, risk assessment, and recovery decision.<n>This work targets a plug-and-play recovery module instead.<n>It introduces a graph-based framework in which a single small language model is trained via reinforcement learning to specialize at each node of a recovery graph.
• Score: 0.5249805590164902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous LLM agents are increasingly deployed in complex real-world workflows, yet they remain vulnerable to runtime behavioral drift, a silent deviation from the original task that can lead to irreversible side effects on external systems. Existing approaches address drift at the prompt level but lack structured mechanisms for step-level detection, risk assessment, and recovery decision. Because the main task-executing agent is often a large and expensive model that cannot be re-trained on every deployment, this work targets a plug-and-play recovery module instead. It introduces a graph-based framework in which a single small language model is trained via reinforcement learning to specialize at each node of a recovery graph, external to the main agent. Each node has a precise role\,: drift classification, operation detection, risk evaluation, or final decision and the model learns to produce structured XML-formatted reasoning adapted to that role. Training combines rule-based structural rewards with an LLM-as-judge semantic-quality signal, so that the model is graded both on how it answers (schema and length) and on what it says. Experiments on the public AppWorld benchmark show that the method generally exploits information about the suspected drift onset to issue correct recovery decisions using a small language model. In addition, the trained small language model reliably respects the prescribed output schema and produces semantically appropriate content in each field according to its assigned node role.

Related papers

• Safe and Adaptive Cloud Healing: Verifying LLM-Generated Recovery Plans with a Neural-Symbolic World Model [9.794020127306473]
  Planning-Aware Semantic self-healing engine, PASE, is a novel fault self-healing framework.<n>Our framework advances autonomous system management by unifying LLM-based reasoning with model-assisted verification and meta-learned guidance.<n> Experiments on a real-world cloud fault injection dataset demonstrate that PASE significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-02T01:45:30Z)
• Divergence Decoding: Inference-Time Unlearning via Auxiliary Models [2.1485350418225244]
  Divergence Decoding (DD) is a mechanism that uses small auxiliary models to steer the logits of the Large Language Models away from specific data during inference.<n>We find the method decisively outperforms state-of-the-art (SOTA) baselines on unlearning benchmarks.
  arXiv  Detail & Related papers  (2026-05-29T13:29:01Z)
• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• In-Context Autonomous Network Incident Response: An End-to-End Large Language Model Agent Approach [3.733830491643117]
  We propose to leverage large language models' pre-trained security knowledge and in-context learning to create an end-to-end agentic solution for incident response planning.<n>Specifically, our agent integrates four functionalities, perception, reasoning, planning, and action, into one lightweight LLM (14b model)<n>Our agent achieves recovery up to 23% faster than those of frontier LLMs.
  arXiv  Detail & Related papers  (2026-02-13T18:09:30Z)
• Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior [58.751981587234916]
  This paper formalizes the Refinement Provenance Inference (RPI) audit task as Refinement Provenance Inference (RPI)<n>We propose RePro, a logit-based framework that fuses teacher-forced likelihood features with logit-ranking signals.<n>During training, RePro learns a transferable representation via shadow fine-tuning, and uses a lightweight linear head to infer provenance on unseen victims without training-data access.
  arXiv  Detail & Related papers  (2026-01-05T10:16:41Z)
• Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs [72.08224879435762]
  textttLearn-to-Ask is a simulator-free framework for learning and deploying proactive dialogue agents.<n>Our approach culminates in the successful deployment of LLMs into a live, large-scale online AI service.
  arXiv  Detail & Related papers  (2025-10-29T12:08:07Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• Augmented Reinforcement Learning Framework For Enhancing Decision-Making In Machine Learning Models Using External Agents [0.0]
  This work proposes a novel technique Augmented Reinforcement Learning framework for the improvement of decision-making capabilities.<n>The external agent can be anyone, like humans or automated scripts, that helps in decision path correction.<n>The framework incorporates two external agents that aid in course correction and the guarantee of quality data at all points of the training cycle.
  arXiv  Detail & Related papers  (2025-08-03T06:17:44Z)
• Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training [18.896813839389893]
  We propose an iterative self-training framework, Agent-R, that enables language Agent to Reflect on the fly.<n>Unlike traditional methods that reward or penalize actions based on correctness, Agent-R leverages MCTS to construct training data that recover correct trajectories from erroneous ones.<n>Our findings demonstrate that Agent-R continuously improves the model's ability to recover from errors and enables timely error correction.
  arXiv  Detail & Related papers  (2025-01-20T11:46:04Z)
• InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance [56.184255657175335]
  We develop textbfInferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.
  arXiv  Detail & Related papers  (2024-01-20T10:41:03Z)
• Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning [56.26889258704261]
  We propose a novel "disentangled" decision-making method, Semantically Aligned task decomposition in MARL (SAMA) SAMA prompts pretrained language models with chain-of-thought that can suggest potential goals, provide suitable goal decomposition and subgoal allocation as well as self-reflection-based replanning. SAMA demonstrates considerable advantages in sample efficiency compared to state-of-the-art ASG methods.
  arXiv  Detail & Related papers  (2023-05-18T10:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.