Runtime-Incremental Transformer for Reinforcement-Learning-Based Adaptive Control

• URL: http://arxiv.org/abs/2609.13545v1
• Date: Fri, 11 Sep 2026 21:16:47 GMT
• Title: Runtime-Incremental Transformer for Reinforcement-Learning-Based Adaptive Control
• Abstract summary: Learning-based adaptive control of robotic manipulators with non-observable friction memory has been addressed by attention-based meta-controllers.<n>The present paper introduces a mechanism that grows and prunes the heads of the attention block during reinforcement learning.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning-based adaptive control of robotic manipulators with non-observable friction memory has been addressed by attention- based meta-controllers whose number of attention heads is fixed before training and is tuned by costly offline search. At long memory horizons, such fixed-capacity controllers are prone to catastrophic failures on a sizeable fraction of training seeds. The present paper introduces a runtime mechanism that grows and prunes the heads of the attention block during reinforcement learning, governed by two signals: the effective rank of the on-policy context distribution, which triggers growth when representational capacity becomes insufficient, and the per-head output magnitude, which flags redundant heads for removal. Policy continuity at growth events and a quantitative bound at prune events are established analytically. On a two- link manipulator with Stribeck friction, the proposed mechanism attains full success across all memory regimes, eliminating the long-horizon failure mode and removing the need for offline tuning of the head count.

Related papers

• Mitigating Bus Bunching with Reinforcement Learning Enhanced by Semantic Stop Embedding [6.0671705325267835]
  Bus bunching degrades service regularity and increases passenger waiting in high-frequency transit.<n>This study introduces an LLM-assisted semantic stop representation for event-driven bus holding control.
  arXiv  Detail & Related papers  (2026-08-10T20:21:52Z)
• Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training [53.3522202888031]
  Adversarial training under long tailed distributions suffers from a dual imbalance: the class imbalance skews the training objective toward head classes.<n>We propose a Confusion Geometry Rebalancing method (CGRm) for long tail adversarial training, a plug in framework that leverages directed robust errors as training signals.
  arXiv  Detail & Related papers  (2026-08-10T14:54:00Z)
• Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability [20.49786007379445]
  We study mechanism-driven detection of training instability by deriving internal monitors from the functional role of each critical module.<n>Our fault-injection experiments on low-precision attention, large learning-rate, and combined faults show that these signals provide distinct signatures for different failures.
  arXiv  Detail & Related papers  (2026-06-26T14:18:22Z)
• Temporal Attention for Adaptive Control of Euler-Lagrange Systems with Unobservable Memory [1.2891210250935148]
  The paper proposes a meta-control architecture in which the gains of a computed-torque controller are generated by a self-attention block.<n>The number of attention heads is selected before policy training through a surrogate analysis of the memory-state gradient.<n>The approach is tested on a 2-DOF manipulator with nonlinear friction and variable payload.
  arXiv  Detail & Related papers  (2026-05-07T19:25:18Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• Training-free Context-adaptive Attention for Efficient Long Context Modeling [57.703159205740185]
  Training-free Context-adaptive Attention (TCA-Attention) is a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference.<n>TCA-Attention achieves a 2.8$times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention.
  arXiv  Detail & Related papers  (2025-12-10T01:54:57Z)
• Verifying Closed-Loop Contractivity of Learning-Based Controllers via Partitioning [52.23804865017831]
  We address the problem of verifying closed-loop contraction in nonlinear control systems whose controller and contraction metric are both parameterized by neural networks.<n>We derive a tractable and scalable sufficient condition for closed-loop contractivity that reduces to checking that the dominant eigenvalue of a symmetric Metzler matrix is nonpositive.
  arXiv  Detail & Related papers  (2025-12-01T23:06:56Z)
• End-to-End Visual Autonomous Parking via Control-Aided Attention [30.52881549605385]
  CAA-Policy is an end-to-end imitation learning system for precise parking.<n>It allows control signal to guide the learning of visual attention via a novel Control-Aided Attention mechanism.
  arXiv  Detail & Related papers  (2025-09-14T04:51:19Z)
• Gated Slot Attention for Efficient Linear-Time Sequence Modeling [59.019501274074564]
  Gated Slot Attention (GSA) enhances Attention with Bounded-memory-Control (ABC) GSA incorporates a gating mechanism inspired by Gated Linear Attention (GLA)
  arXiv  Detail & Related papers  (2024-09-11T09:49:50Z)
• From Imitation to Refinement -- Residual RL for Precise Assembly [19.9786629249219]
  Recent advances in Behavior Cloning (BC) have made it easy to teach robots new tasks.<n>However, we find that the ease of teaching comes at the cost of unreliable performance.<n>We devise a simple yet effective method, ResiP, that overcomes the reliability problem while retaining BC's ease of teaching and long-horizon capabilities.
  arXiv  Detail & Related papers  (2024-07-23T17:44:54Z)
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning [63.93193829913252]
  We propose an innovative METL strategy called SHERL for resource-limited scenarios. In the early route, intermediate outputs are consolidated via an anti-redundancy operation. In the late route, utilizing minimal late pre-trained layers could alleviate the peak demand on memory overhead.
  arXiv  Detail & Related papers  (2024-07-10T10:22:35Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.