StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments

• URL: http://arxiv.org/abs/2608.24804v1
• Date: Tue, 25 Aug 2026 16:48:05 GMT
• Title: StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments
• Abstract summary: We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed.<n>The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration.
• Score: 9.081306845678679
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed. The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration. StarHarness constructs a compact evolution pool by stratifying tasks according to baseline failure behavior, separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks for evaluating generalization. Across ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, harness evolution improves full-benchmark performance by 20-35 percentage points over the default harness after 4-12 accepted changes per environment. These gains persist on tasks excluded from evolution and transfer without re-evolution across GPT and Qwen model families. Trace analysis links the improvements to interface repairs, environment conventions, and operational knowledge that compresses search, with fewer false-positive diagnoses and shorter trajectories in several settings. StarHarness therefore offers a practical way to reduce persistent model-environment mismatch in tool-rich enterprise tasks.

Related papers

• ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement [46.36989489677189]
  We propose a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution.<n> ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies.<n>It decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection.
  arXiv  Detail & Related papers  (2026-09-14T00:10:45Z)
• MOAE: Multi-Objective Agent Evolution with Pareto-Preserving Search [5.257073134713488]
  Multi-Objective Agent Evolution organizes iterative in-context refinement as a Pareto-preserving evolutionary search over complete agent rollouts.<n>We show that MOAE consistently improves task performance and trajectory quality while maintaining strong safety under matched rollout budgets.
  arXiv  Detail & Related papers  (2026-09-05T09:31:34Z)
• Self-Evolving Embodied Agents via Skill-Harness Evolution [53.307052568223945]
  Embodied agents are increasingly built as systems around foundation models.<n>While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs.<n>We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen.
  arXiv  Detail & Related papers  (2026-08-11T18:55:58Z)
• Evo-Bench: Can Language Models Improve Agent Harness? [44.72089415085898]
  Evo-Bench is the first benchmark designed to evaluate models' intrinsic harness-evolving capabilities across Search, Office, and General agent domains.<n>Top models achieve massive absolute gains reaching 16.6 points, closely approaching state-of-the-art human-engineered baselines.
  arXiv  Detail & Related papers  (2026-08-10T03:49:28Z)
• EvolveNet: Collaborative Harness Evolution for Agent Self-Improvement [70.68218215070745]
  Existing approaches assume that all execution experience can be routed to a single harness.<n>We introduce EvolveNet, a paradigm of harness evolution that moves experience extraction to the data.
  arXiv  Detail & Related papers  (2026-08-05T15:37:18Z)
• OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution [71.06848799741152]
  AI agents operate in persistent environments where early state changes can influence decisions far into the future.<n>Current safety benchmarks often fail to capture cumulative risks because they focus on short, static tasks.<n>We introduce OpenART, an open-ended arena for scalable agent red teaming through environment evolution.
  arXiv  Detail & Related papers  (2026-08-01T13:51:55Z)
• Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams [41.672123164003814]
  We introduce Adaptive Auto-Harness, a framework and system for such streams.<n>It decomposes the gap to an oracle harness into evolution loss and adaptation loss.<n>It outperforms five existing auto-harness baselines across prediction-market, security-competition, and event-forecasting streams.
  arXiv  Detail & Related papers  (2026-06-01T06:51:14Z)
• Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses [57.20181537213498]
  Agentic Harness Engineering (AHE) is a closed loop that automates harness engineering.<n>AHE addresses challenges through three matched observability pillars.<n>Ten AHE lift pass@1 on Terminal-Bench 2 from 69.7% to 77.0%, surpassing the human-designed harness Codex-CLI.<n>On SWE-bench-verified it tops aggregate success at 12% fewer tokens than the seed, and on Terminal-Bench 2 it yields +5.1 to +10.1pp cross-family gains.
  arXiv  Detail & Related papers  (2026-04-28T16:55:02Z)
• ANCHOR: Branch-Point Data Generation for GUI Agents [52.22377425487]
  End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data.<n>We present a trajectory expansion framework Anchor that bootstraps scalable desktop supervision from a small set of verified seed demonstrations.<n>Experiments on standard desktop benchmarks, OSWorld and WindowsAgentArena, show that models fine-tuned on our expanded corpus achieve consistent improvements.
  arXiv  Detail & Related papers  (2026-02-06T19:55:26Z)
• PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution [64.15555230987222]
  PACEvolve is a framework designed to robustly govern the agent's context and search dynamics.<n>We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement.
  arXiv  Detail & Related papers  (2026-01-15T18:25:23Z)
• HiVA: Self-organized Hierarchical Variable Agent via Goal-driven Semantic-Topological Evolution [13.440964262446558]
  Hierarchical Variable Agent (HiVA) is a novel framework modeling agentic as self-organized graphs with the Semantic-Topological Evolution (STEV) algorithm.<n> Experiments on dialogue, coding, Longcontext Q&A, mathematical, and agentic benchmarks demonstrate improvements of 5-10% in task accuracy and enhanced resource efficiency.
  arXiv  Detail & Related papers  (2025-08-29T18:51:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.