Closed-Loop Control with Rule-Aligned Small Language Models and Multi-Agent Self-Correction

• URL: http://arxiv.org/abs/2607.09713v1
• Date: Wed, 24 Jun 2026 13:49:01 GMT
• Title: Closed-Loop Control with Rule-Aligned Small Language Models and Multi-Agent Self-Correction
• Abstract summary: This work investigates whether a compact Small Language Model (SLM) can be retrained for control reasoning and embedded in a validator-guided correction loop.<n>In randomized thermal-control simulations (30 experiments with 500 steps each), the framework achieves 91.5% average action-alignment accuracy.<n>Under symbolic re-mapping, it maintains a 95% in-range rate, indicating robust physical regulation despite reduced token-level agreement.
• Score: 17.527470001317685
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A key step toward autonomous industrial operation is the ability to create and reconfigure control policies from natural-language requirement specifications, with minimal or no manual redesign. In this setting, policy generation by AI agents can be a credible path when paired with a plant-aware validator (e.g., a digital twin) that can check generated candidate actions before execution. However, practical deployment is constrained by inference latency and compute footprint: large cloud-based models are often too slow, opaque, or data-sensitive for edge closed-loop use. This work investigates whether a compact Small Language Model (SLM) can be retrained for control reasoning and embedded in a validator-guided correction loop. We use a Qwen2.5-1.5B model aligned via Group Relative Policy Optimization (GRPO), combined with (i) an action agent, (ii) a symbolic/digital-twin-style validation layer, and (iii) a reprompting agent that iteratively steers outputs toward valid actions. In randomized thermal-control simulations (30 experiments with 500 steps each), the framework achieves 91.5% average action-alignment accuracy (86.3%--100% across cases) at 3.84\,s mean inference latency. Under symbolic re-mapping, it maintains a 95% in-range rate, indicating robust physical regulation despite reduced token-level agreement. These results support SLM+validator architectures as a practical path toward reconfigurable autonomous control at the edge.

Related papers

• Schema-Adaptive Action-Conditioned JEPA for Cross-Machine CNC Transfer under Partial Sensor Overlap [0.0]
  Cross-machine deployment of industrial world models requires transfer across changes in dynamics, sensing interfaces, sampling regimes, and control units.<n>We study a schema-adaptive action-conditioned Joint-Embedding Predictive Architecture (SAAC-JEPA) for CNC dynamics.<n> Evaluation uses group-disjoint source splits, source-only normalization, held-out self-supervised validation, unit audits, and a sealed target test after model locking.
  arXiv  Detail & Related papers  (2026-09-13T15:10:06Z)
• AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning [10.408251801653723]
  Large language models (LLMs) achieve strong reasoning performance, which depends critically on time decisions.<n>Recent adaptive methods partially address this limitation, but they primarily adapt either recommended decodingity (how the model explores) or reasoning compute (how long the model reasons) in isolation.<n>We introduce a within-trajectory joint control view, and instantiate it in AutoCRAT.
  arXiv  Detail & Related papers  (2026-08-30T19:24:58Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• From Detection to Action: Using LLM Agents for Fault-Tolerant Control [0.21313236831585222]
  We propose an agentic Large Language Model (LLM) framework for Fault-Tolerant Control (FTC)<n>It transforms fault detection outputs into constraint-aware recovery actions grounded in plant-specific knowledge.<n>We show that semantically grounded agents can derive valid recovery decisions within latency budgets compatible with the respective process dynamics.
  arXiv  Detail & Related papers  (2026-06-26T12:13:58Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Don't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM Agents [0.0]
  Inference-time compute scaling has emerged as a powerful technique for improving the reliability of large language model (LLM) agents.<n>We introduce TrACE, a training-free controller that allocates LLM calls adaptively across agent timesteps by measuring inter-rollout action agreement.
  arXiv  Detail & Related papers  (2026-04-09T15:34:22Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute [10.548368675645403]
  CoRefine is a confidence-guided self-refinement method that achieves competitive accuracy using a fraction of the tokens.<n>The controller consumes full-trace confidence to decide whether to halt, re-examine, or try a different approach.<n>We extend this to CoRefine-Tree, a hybrid sequential-parallel variant that adaptively balances exploration and exploitation.
  arXiv  Detail & Related papers  (2026-02-09T17:44:41Z)
• ProAct: Agentic Lookahead in Interactive Environments [56.50613398808361]
  ProAct is a framework that enables agents to internalize accurate lookahead reasoning through a two-stage training paradigm.<n>We introduce Grounded LookAhead Distillation (GLAD), where the agent undergoes supervised fine-tuning on trajectories derived from environment-based search.<n>We also propose the Monte-Carlo Critic (MC-Critic), a plug-and-play auxiliary value estimator designed to enhance policy-gradient algorithms.
  arXiv  Detail & Related papers  (2026-02-05T05:45:16Z)
• Reasoning is a Modality [4.055765634948606]
  We study abstract reasoning, an ability central to human intelligence.<n>Modern AI systems operate as sequence-of-behavior prediction machines.<n>Humans can explain an action by decoding internal state, while AI systems can produce fluent post-hoc rationalizations.
  arXiv  Detail & Related papers  (2026-01-20T03:37:17Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• On Geometric Structures for Policy Parameterization in Continuous Control [7.056222499095849]
  We propose a novel, computationally efficient action generation paradigm that preserves the structural benefits of operating on a unit manifold.<n>Our method decomposes the action into a deterministic directional vector and a learnable concentration, enabling efficient between the target direction and uniform noise.<n> Empirically, our method matches or exceeds state-of-the-art methods on standard continuous control benchmarks.
  arXiv  Detail & Related papers  (2025-11-11T13:32:38Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)
• Actor-Critic based Improper Reinforcement Learning [61.430513757337486]
  We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process. We propose two algorithms: (1) a Policy Gradient-based approach; and (2) an algorithm that can switch between a simple Actor-Critic scheme and a Natural Actor-Critic scheme.
  arXiv  Detail & Related papers  (2022-07-19T05:55:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.