Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable

• URL: http://arxiv.org/abs/2607.13285v1
• Date: Tue, 14 Jul 2026 21:39:55 GMT
• Title: Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
• Abstract summary: We introduce the Harness Handbook, a behavior-contextcentric representation synthesized automatically from a harness.<n>We also introduce Behavior-Guided Progressive Disclosure (BGPD), which guides agents from high-level behaviors to relevant implementation details.<n>On diverse modification requests from two open-source harnesses, Handbook-Assisted planning improves behavior localization and edit-plan quality.
• Score: 46.2784020602283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long-context processing ease inspection, but still leave this behavior-to-code mapping to be recovered by hand. Behavior localization is therefore a central bottleneck in harness evolution. We introduce the Harness Handbook, a behavior-centric representation synthesized automatically from a harness codebase via static analysis and LLM-assisted structuring, linking each behavior to its corresponding source. We also introduce Behavior-Guided Progressive Disclosure (BGPD), which guides agents from high-level behaviors to relevant implementation details and verifies candidate locations against the current source. On diverse modification requests from two open-source harnesses, Handbook-Assisted planning improves behavior localization and edit-plan quality while using fewer planner tokens, with the largest gains on scattered sites, rarely executed paths, and cross-module interactions. Evolving complex agentic systems thus depends not only on generating edits, but also on determining where those edits should be made.

Related papers

• Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback [0.0]
  Large Language Model (LLM) agents face a fundamental tension: the agent's behavior is frozen at deployment time, while the business rules and edge cases it must handle continue to evolve.<n>Existing approaches address agent construction and one-time evaluation but provide no structured mechanism for continuous post-deployment behavioral correction.<n>We introduce Agent Gym, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop.
  arXiv  Detail & Related papers  (2026-08-16T07:27:26Z)
• AdaptAgent: A Multi-agent, Domain-Guided Reasoning Framework for Code Adaptation [14.734454356396157]
  We present AdaptAgent, a multi-agent, domain-guided reasoning framework for code adaptation.<n>Rather than relying on single-shot prompting, AdaptAgent decomposes adaptation into specialized agents that communicate via typed artifacts.<n>On a real-world dataset, AdaptAgent outperforms strong baselines in semantic correctness and produces patches that mirror developers' actual adaptation patterns.
  arXiv  Detail & Related papers  (2026-08-05T05:34:40Z)
• Formal Verification of Agentic Systems over Operational Data [59.98246281888422]
  Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world where they act on persistent operational data.<n>Existing approaches do not provide such system-level guarantees.<n>We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data.
  arXiv  Detail & Related papers  (2026-08-04T13:01:30Z)
• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents [1.4323566945483497]
  Foundation models have transformed automated code generation, yet autonomous software-engineering agents remain unreliable in realistic development settings.<n>We propose a model-harness-environment system in which a runtime substrate mediates how a foundation-model agent observes a project, acts on it, receives feedback, and establishes that a change is complete.<n>The framework reframes the central question of autonomous software engineering from whether a foundation model can produce a patch to whether the model-harness-environment system can produce a verifiably correct, attributed, and maintainable change.
  arXiv  Detail & Related papers  (2026-05-13T11:14:59Z)
• Agent Mentor: Framing Agent Knowledge through Semantic Trajectory Analysis [4.365760422569902]
  We introduce an analytics pipeline implemented as part of the Agent Mentor open-source library.<n>It monitors and incrementally adapts the system prompts defining another agent's behavior.<n>The pipeline improves performance by systematically injecting corrective instructions into the agent's knowledge.
  arXiv  Detail & Related papers  (2026-04-12T08:02:54Z)
• ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibration [68.89572566071575]
  ETAgent is a training framework for calibrating agent's tool-use behavior.<n>It is designed to progressively calibrate erroneous behavioral patterns to optimal behaviors.
  arXiv  Detail & Related papers  (2026-01-11T11:05:26Z)
• Data-Model Co-Evolution: Growing Test Sets to Refine LLM Behavior [10.041741229516141]
  Large Language Models (LLMs) allow developers to govern model behavior by editing prompt instructions.<n>We operationalize this paradigm in an interactive system designed to address the challenge of encoding subtle, domain-specific policies into prompt instructions.
  arXiv  Detail & Related papers  (2025-10-14T17:07:37Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.