HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning

• URL: http://arxiv.org/abs/2609.13739v1
• Date: Sat, 12 Sep 2026 06:22:45 GMT
• Title: HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning
• Abstract summary: We develop HarnessBandit, an online scheduler that selects one harness per step.<n>We train Qwen3.5-2B across six harnesses on ClawGym and evaluate on PinchBench and ClawEval.
• Score: 40.61476978001808
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-model agents are increasingly deployed through diverse harnesses that differ in system prompts, tool schemas, control loops, and trajectory formats. The same model can perform unevenly across these interfaces, making robustness to harness variation an important objective. A natural approach is to train a shared policy through multiple harnesses, but doing so introduces a scheduling problem: each training step should favor a harness that currently provides a useful learning signal while also producing an update that benefits the other harnesses. We develop HarnessBandit, an online scheduler that selects one harness per optimizer step. After a group-relative policy optimization (GRPO) update, it observes learnability -- the mean absolute advantage on the batch -- and transferability -- the cosine between a low-dimensional gradient sketch of the current harness and exponential moving averages of the remaining harnesses. The two signals are fused after pooled sliding-window min-max normalization and sampled with a visit-dependent bonus and an explicit exploration floor. We train Qwen3.5-2B across six harnesses on ClawGym and evaluate on PinchBench (held-out tasks, in-distribution OpenClaw) and ClawEval (held-out tasks and harness). HarnessBandit improves over mixed-batch multi-harness training on both benchmarks, while training diagnostics indicate that learnability and transferability provide distinct, evolving signals.

Related papers

• WHALE: A Simple Recipe for Joint Harness-Weight Optimization [48.550149284101536]
  Weight-Harness Alternating LEarning (WHALE) is a recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model.<n>Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy.
  arXiv  Detail & Related papers  (2026-08-31T18:12:55Z)
• Harness Continual Learning: Continual Adaptation Beyond Model Parameters [9.54786133734043]
  We formulate a new continual learning paradigm in which the harness evolves around a frozen foundation model.<n>A candidate commits a harness only after checking current improvement, historical retention, and validity.<n>Experiments demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings.
  arXiv  Detail & Related papers  (2026-08-19T15:12:31Z)
• Self-Evolving Embodied Agents via Skill-Harness Evolution [53.307052568223945]
  Embodied agents are increasingly built as systems around foundation models.<n>While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs.<n>We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen.
  arXiv  Detail & Related papers  (2026-08-11T18:55:58Z)
• Confusion-Geometry Rebalancing for Long-Tailed Adversarial Training [53.3522202888031]
  Adversarial training under long tailed distributions suffers from a dual imbalance: the class imbalance skews the training objective toward head classes.<n>We propose a Confusion Geometry Rebalancing method (CGRm) for long tail adversarial training, a plug in framework that leverages directed robust errors as training signals.
  arXiv  Detail & Related papers  (2026-08-10T14:54:00Z)
• HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry [35.87794858139959]
  HarnessX is a foundry for composable, adaptive, and evolvable agent harnesses.<n>It assembles typed harness primitives via a substitution algebra, adapts them through AEGIS.<n>It closes the harness-model loop by turning trajectories into both harness updates and model training signal.
  arXiv  Detail & Related papers  (2026-06-12T08:27:11Z)
• HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness [80.53160974102161]
  We introduce HarnessBridge, a lightweight learnable harness controller that parameterizes the agent--environment interface as a bidirectional projection.<n>On Terminal-Bench2.0 and SWE-bench Verified, HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length.
  arXiv  Detail & Related papers  (2026-06-11T04:18:37Z)
• ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control [14.748861060405973]
  Reinforcement Learning (RL) post-training has become the standard for aligning generative models with human preferences.<n>The prevailing practice of early scalarization'' collapses rewards into a fixed weighted sum when multiple criteria matter.<n>This commits the model to a single trade-off point at training time, providing no inference-time control over inherently conflicting goals.
  arXiv  Detail & Related papers  (2026-04-22T17:44:56Z)
• Learning Expected Emphatic Traces for Deep RL [32.984880782688535]
  Off-policy sampling and experience replay are key for improving sample efficiency and scaling model-free temporal difference learning methods. We develop a multi-step emphatic weighting that can be combined with replay, and a time-reversed $n$-step TD learning algorithm to learn the required emphatic weighting.
  arXiv  Detail & Related papers  (2021-07-12T13:14:03Z)
• Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
  Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness. We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
  arXiv  Detail & Related papers  (2020-10-26T04:44:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.