JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution

• URL: http://arxiv.org/abs/2608.25593v2
• Date: Thu, 03 Sep 2026 15:46:32 GMT
• Title: JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
• Abstract summary: JIT-Agent is a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly.<n>JIT-Agent is the first model purpose-built for just-in-time harness generation.
• Score: 74.58840188662151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliable execution, and self-evolve by distilling performance signals from an expanding archive of prior harness configurations. Equipped with JIT-Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA (+9.1) and OdysseyBench (+4.3), while the already strong GLM-5.2 gains up to +20.2 points. Across controlled evaluations, JIT-Agent-generated harnesses are performance-competitive with mature agent runtimes such as OpenCode and Claude Code and consistently improve multi-scale model families of DeepSeek V4, Mimo-V2.5, and Qwen3.6. To our knowledge, JIT-Agent is the first model purpose-built for just-in-time harness generation, establishing harness intelligence as a trainable, transferable, and compounding dimension of agent capability orthogonal to model scaling.

Related papers

• Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails [23.287109132562403]
  Automated harness evolution can enable smaller models to perform well on domain-specific tasks at a fraction of frontier-model cost.<n>We show that imitation transfers knowledge and increases scaffold usage, but disrupts model-harness fit.<n>We develop an on-policy expert-correction pipeline, automated by a meta-level MLE agent, that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn.
  arXiv  Detail & Related papers  (2026-09-08T17:53:49Z)
• Prime Agent: A Self-Improving RLM Harness [5.540645425386541]
  Prime Agent is an open-source harness for long-horizon evaluation and coding-agent.<n>It standardizes execution, recovery, verification, and resource accounting while leaving strategy construction to the model.<n>Prime Agent raises ARC-AGI-3 RHAE Best@1 from 30% to 95.5% and matches or exceeds native and popular harnesses.
  arXiv  Detail & Related papers  (2026-08-24T17:54:19Z)
• Agent Lightning v1.0: Towards Harnessed Agentic RL [28.087691629023038]
  We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL.<n>It supports arbitrary agent harnesses and serves as a practical testbed for studying these challenges.<n>We evaluate it on instruction-following, search, and coding agents, and provide a complete reproducible pipeline for coding-agent RL.
  arXiv  Detail & Related papers  (2026-08-18T08:50:13Z)
• HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry [35.87794858139959]
  HarnessX is a foundry for composable, adaptive, and evolvable agent harnesses.<n>It assembles typed harness primitives via a substitution algebra, adapts them through AEGIS.<n>It closes the harness-model loop by turning trajectories into both harness updates and model training signal.
  arXiv  Detail & Related papers  (2026-06-12T08:27:11Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? [71.21547572568655]
  AgenTracer-8B is a lightweight failure tracer trained with multi-granular reinforcement learning.<n>On the Who&When benchmark, AgenTracer-8B outperforms giant proprietary LLMs like Gemini-2.5-Pro and Claude-4-Sonnet by up to 18.18%.<n>AgenTracer-8B delivers actionable feedback to off-the-shelf multi-agent systems like MetaGPT and MaAS with 4.8-14.2% performance gains.
  arXiv  Detail & Related papers  (2025-09-03T13:42:14Z)
• R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science [70.1638335489284]
  High-level machine learning engineering tasks remain labor-intensive and iterative.<n>We introduce R&D-Agent, a comprehensive, decoupled, and framework that formalizes the machine learning process.<n>R&D-Agent defines the MLE into two phases and six components, turning agent design for MLE into a principled, testable process.
  arXiv  Detail & Related papers  (2025-05-20T06:07:00Z)
• DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agents [31.542621203252295]
  Large language model (LLM)-powered agents are increasingly used in recommender systems (RSs)<n>This paper presents the first systematic investigation of memory-based vulnerabilities in LLM-powered recommender agents.<n>We propose a novel black-box attack framework named DrunkAgent.
  arXiv  Detail & Related papers  (2025-03-31T07:35:40Z)
• xLAM: A Family of Large Action Models to Empower AI Agent Systems [111.5719694445345]
  We release xLAM, a series of large action models designed for AI agent tasks. xLAM consistently delivers exceptional performance across multiple agent ability benchmarks.
  arXiv  Detail & Related papers  (2024-09-05T03:22:22Z)
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
  We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR) In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle. In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-02-27T12:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.