Daily Paper Briefing

Abstract Overview

This paper presents JIT-Agent, a model trained to generate task-adaptive agent harnesses on demand rather than relying on a fixed, manually engineered scaffold. The authors formalize an agent harness as four executable modules—memory, planning, action, and capability orchestration—and build a unified codebase, HarnessFactory, to support training and evaluation. JIT-Agent is trained in three stages: task-conditioned customization, repair from failed harness generations, and online evolution using reward, latency, and cost signals. Experiments across deep research, daily work, planning, and workspace benchmarks indicate that changing the harness can substantially improve the performance of the same underlying backbone model.

Novelty

The distinctive contribution is a learned just-in-time harness generation paradigm: instead of optimizing a durable harness ahead of time, JIT-Agent synthesizes an instance-specific executable harness at inference time and can update an archive of better harnesses from feedback. The paper also frames this as "harness intelligence" and operationalizes it with a fixed four-module protocol plus a three-stage training pipeline that includes learned repair and evolutionary optimization.

Results

Across all 18 directly matched backbone–benchmark pairs reported for GLM-5.2 and DeepSeek-V4-Flash, JIT-generated harnesses improved over the default scaffold, raising the nine-benchmark average from 74.1 to 81.8 for GLM-5.2 and from 66.7 to 75.5 for DeepSeek-V4-Flash. In the main benchmark table, JIT-equipped systems achieve the best score in eight of nine columns, and controlled comparisons show performance competitive with strong fixed harnesses such as OpenCode and Claude Code. The controlled studies also report the lowest token consumption and API cost for JIT-Agent in all six tested settings, indicating that gains are not simply due to longer or more expensive trajectories.

Key Points

1. JIT-Agent models the agent harness as a composable four-module executable artifact and generates task-specific harnesses for arbitrary off-the-shelf agentic LLMs.
2. The training pipeline combines supervised harness customization, bounded repair from execution failures, and Evo-GDPO-based online evolution using reward, latency, and cost signals.
3. Empirically, JIT-generated harnesses consistently improve fixed backbones across benchmarks and often offer better cost–performance trade-offs than reusable fixed harnesses.

References

• arXiv: https://arxiv.org/abs/2608.25593v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2608.25593v1
• Hugging Face Papers: https://huggingface.co/papers/2608.25593
• GitHub: https://github.com/bingreeky/JIT
• Project: https://bingreeky.github.io/JIT-site