AI Training Manager: Bounded Closed-Loop Control of Adaptive Training Recipes

• URL: http://arxiv.org/abs/2606.29871v1
• Date: Mon, 29 Jun 2026 07:07:33 GMT
• Title: AI Training Manager: Bounded Closed-Loop Control of Adaptive Training Recipes
• Abstract summary: Manager reads structured telemetry snapshots from an active run, audits a constrained action space, and returns validated updates to training parameters.<n>On TinyStories, the manager detects and corrects overfitting, achieving a validation loss 60% lower than the baseline.<n>In a robotic manipulation reinforcement-learning task, we use the same bounded decision interface in an episodic closed-loop setting, where manager updates are applied at evaluation or checkpoint boundaries.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the AI Training Manager, a bounded LLM-based supervisory controller for adaptive machine learning training. Standard training pipelines often rely on fixed recipes or single-axis schedulers, which can struggle with mid-run failures such as severe overfitting, loss imbalance, exploration collapse, or unsafe exploration. Rather than replacing mathematical optimizers or acting as an unconstrained coding agent, the manager operates through a schema-conditioned interface: it reads structured telemetry snapshots from an active run, audits a constrained action space, and returns validated updates to training parameters such as learning rate, regularization strength, loss-weight coefficients, and exploration settings. We evaluate this architecture across supervised language modeling and reinforcement learning. On TinyStories, the manager detects and corrects overfitting, achieving a validation loss 60% lower than the baseline while producing auditable intervention logs. In this supervised setting, we additionally show that manager inference does not need to block the training loop: training can continue while a manager response is pending, and validated updates can be applied asynchronously once available. In a robotic manipulation reinforcement-learning task, we use the same bounded decision interface in an episodic closed-loop setting, where manager updates are applied at evaluation or checkpoint boundaries. The manager mitigates both conservative and unsafe exploration regimes. These results suggest that schema-conditioned LLMs can serve as bounded supervisory managers for live training runs, complementing conventional optimizers and schedulers with interpretable, multi-axis intervention capabilities

Related papers

• Training Language Models to Cooperate with Inference-Time Controllers [25.4633535418324]
  CALM (Controller-Aware Language Models) is a post-training framework that explicitly places controllers in the training loop.<n>We formulate controller-aware post-training as multi-task reinforcement learning over controller-induced interaction protocols.<n>We evaluate CALM on held-out controller compositions and broader controller shifts, showing that controller-aware post-training improves generalization across inference-time beyond single-controller optimization.
  arXiv  Detail & Related papers  (2026-07-26T17:44:33Z)
• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents [51.523913302114266]
  TRACE (Turn-level Reward Assignment via Credit Estimation) is a dense credit-assignment method for agentic reinforcement learning.<n>It represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values.<n>It substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data.
  arXiv  Detail & Related papers  (2026-07-15T16:16:42Z)
• Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning [0.05371337604556311]
  Large language model (LLM) agents are usually improved by changing prompts, calibrated models, or hand-written, while the execution harness around the model is treated as fixed infrastructure.<n>We formalize harness operation as a finite-horizon Harness MDP, where a lightweight controller selects structural execution actions while the LLM executor remains frozen.
  arXiv  Detail & Related papers  (2026-07-05T22:11:18Z)
• Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It [23.538941671680806]
  Tool use enables large language models to perform complex tasks.<n>Recent agentic reinforcement learning methods show promise for enhancing model capabilities.<n>Some models exhibit catastrophic collapse, where performance abruptly drops and tool-invocation structures fail.
  arXiv  Detail & Related papers  (2026-06-24T16:55:56Z)
• PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents [57.69188221227875]
  We propose a Privileged trAce Co-Training framework for multi-turn tool-use agents.<n>The key idea is to use expert traces only as training-time optimization signals rather than rollout-time hints.<n>We show that PACT consistently improves over strong SFT- and RL-based baselines.
  arXiv  Detail & Related papers  (2026-06-15T04:46:23Z)
• Harnessing LLM Agents with Skill Programs [58.356514745548026]
  We introduce HASP, a new framework that upgrades skills into executable Program Functions (PFs)<n>PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context.<n> HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks.
  arXiv  Detail & Related papers  (2026-05-18T01:35:11Z)
• Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training [50.86545293331458]
  Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from.<n>We present a simple yet effective batch-adaptive objective that replaces fixed clipping with the normalized effective sample size of the policy ratios.
  arXiv  Detail & Related papers  (2026-05-12T16:44:47Z)
• SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents [0.31034395048547575]
  Existing approaches rely on static prompting strategies or handcrafteds to select actions such as code editing, file navigation, and test execution.<n>We propose SWE-Shepherd, a framework that introduces Process Reward Models (PRMs) to provide dense, step-level supervision for repository-level code agents.<n>Experiments on SWE-Bench Verified demonstrate improved interaction efficiency and action quality, while also highlighting challenges in aligning intermediate rewards with final task success.
  arXiv  Detail & Related papers  (2026-04-12T06:51:47Z)
• Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous Vehicles [40.195332742819865]
  This study proposes an instruction-realization framework that leverages a large language model (LLM) to interpret instructions.<n>It generates executable scripts that schedule multiple model predictive control (MPC)-based motion planners based on real-time feedback, and converts planned trajectories into control signals.
  arXiv  Detail & Related papers  (2026-04-09T09:32:21Z)
• Double Check My Desired Return: Transformer with Target Alignment for Offline Reinforcement Learning [64.6334337560557]
  Reinforcement learning via supervised learning (RvS) frames offline RL as a sequence modeling task.<n>Decision Transformer (DT) struggles to reliably align the actual achieved returns with specified target returns.<n>We propose Doctor, a novel approach that Double Checks the Transformer with target alignment for Offline RL.
  arXiv  Detail & Related papers  (2025-08-22T14:30:53Z)
• Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt [80.43623986759691]
  We introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD. The framework equips the UAD with continual learning capability through contrastively-learned prompts. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation.
  arXiv  Detail & Related papers  (2024-01-02T03:37:11Z)
• SMART: Self-supervised Multi-task pretrAining with contRol Transformers [34.604339091596884]
  Self-supervised pretraining has been extensively studied in language and vision domains. It is difficult to properly design such a pretraining approach for sequential decision-making tasks. We propose a generic pretraining framework for sequential decision making.
  arXiv  Detail & Related papers  (2023-01-24T05:01:23Z)
• URLB: Unsupervised Reinforcement Learning Benchmark [82.36060735454647]
  We introduce the Unsupervised Reinforcement Learning Benchmark (URLB) URLB consists of two phases: reward-free pre-training and downstream task adaptation with extrinsic rewards. We provide twelve continuous control tasks from three domains for evaluation and open-source code for eight leading unsupervised RL methods.
  arXiv  Detail & Related papers  (2021-10-28T15:07:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.