HARBOR: Automated Harness Optimization

• URL: http://arxiv.org/abs/2604.20938v1
• Date: Wed, 22 Apr 2026 13:45:12 GMT
• Title: HARBOR: Automated Harness Optimization
• Abstract summary: We argue that harness design is a first-class machine-learning problem.<n>We give a reference solver, HARBOR, built from a block-additive SAAS surrogate, multi-fidelity cost-aware acquisition, and TuRBO trust regions.
• Score: 2.7324157162184157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon language-model agents are dominated, in lines of code and in operational complexity, not by their underlying model but by the harness that wraps it: context compaction, tool caching, semantic memory, trajectory reuse, speculative tool prediction, and the glue that binds the model to a sandboxed execution environment. We argue that harness design is a first-class machine-learning problem and that automated configuration search dominates manual stacking once the flag space exceeds a handful of bits. We defend this claim in two steps. First, we formalize automated harness optimization as constrained noisy Bayesian optimization over a mixed-variable, cost-heterogeneous configuration space with cold-start-corrected rewards and a posterior chance-constrained safety check, and give a reference solver, HARBOR (Harness Axis-aligned Regularized Bayesian Optimization Routine), built from a block-additive SAAS surrogate, multi-fidelity cost-aware acquisition, and TuRBO trust regions. Second, we instantiate the problem in a flag-gated harness over a production coding agent and report a controlled four-round manual-tuning case study against a fixed task suite and an end-to-end HARBOR run. The formulation itself is task-class agnostic: the configuration space, reward correction, acquisition, and safety check apply to any agent harness with a bounded flag space and a reproducible task suite.

Related papers

• AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces [61.51487329124135]
  AutoSaddler is a framework that formulates harness improvement as an offline learning problem.<n>It iteratively updates the harness using failure signals from mini-batches.<n>Experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 show that AutoSaddler substantially improves agent performance.
  arXiv  Detail & Related papers  (2026-08-24T09:45:08Z)
• ClawGym II: Exploring Black-Box RL on Agent Harness [82.92963070856416]
  Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment.<n>We present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses.
  arXiv  Detail & Related papers  (2026-08-17T16:53:03Z)
• SBCO: Self-Supervised, Verifier-Grounded Harness Optimization For Planning Agents [8.206010431605044]
  Self-improving agents seek to reduce the human engineering effort behind AI systems by enabling them to evolve and self-improve their performance over time.<n>We introduce SBCO, a verifier-grounded harness in the same closed-loop, improve-from-experience family as the Gdel-machine methods, but self-supervised rather than self-referential.
  arXiv  Detail & Related papers  (2026-08-10T19:25:59Z)
• HarnessOpt-Bench: Evaluating LLMs at Harness Optimization [3.846854817984248]
  We introduce a benchmark for end-to-end harness optimization under expensive evaluation regimes.<n>A trusted execution environment enforces the evaluation boundary, target-agent resource use, and preserves candidate versions for audit.<n>Experiment results show that models separate more than the coding harnesses they act through, native harnesses are not consistently superior, and gains vary substantially across tasks.
  arXiv  Detail & Related papers  (2026-08-06T17:21:05Z)
• SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant [79.24089819400126]
  Subsampled TurboQuant (SSTQ) is a framework that combines overcomplete equal-width tight frames, coordinate subsampling, and privacy-aware quantization.<n>SSTQ achieves mean squared error scaling while using only $lceil log N il + b$ bits per client, where $N = (d)$ is the frame size.<n>We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4b)$ to $O(2b)$.
  arXiv  Detail & Related papers  (2026-08-05T17:51:25Z)
• A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain [2.452218069720666]
  Two 2026 systems, Meta-Harness and HyperAgents, show that this harness can itself be optimized or even self-rewritten by an agentic proposer.<n>We take a narrower, more constrained position: treat the harness as a small, fixed, human-legible action space and learn a policy over it online.<n>We instantiate this control system with DSPy as both the context assembler and the source of the strongest non-adaptive baseline.
  arXiv  Detail & Related papers  (2026-07-28T08:10:25Z)
• MemoHarness: Agent Harnesses That Learn from Experience [46.167787606315585]
  We introduce MemoHarness, an adaptive harness optimization framework that learns from its own executions.<n>MemoHarness decomposes the harness into six editable control dimensions, stores per-case diagnoses and distilled global patterns in a dual-layer experience bank.<n>In our evaluation across shell-agent, code-generation, and analytical-reasoning benchmarks, MemoHarness improves over the fixed harnesses we compare against and shows selective transfer to unseen suites and base models.
  arXiv  Detail & Related papers  (2026-07-14T21:22:18Z)
• Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference [51.55077364626896]
  We introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimize agent harness using only past trajectories.<n>RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel.<n>A single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading.
  arXiv  Detail & Related papers  (2026-06-04T09:26:00Z)
• Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization [27.280175507074322]
  We propose a subspace control framework for constrained model training.<n>SIFT (spectral interference-free training) consistently achieves substantial and robust performance improvements across all tasks.
  arXiv  Detail & Related papers  (2026-04-05T19:27:24Z)
• SEVerA: Verified Synthesis of Self-Evolving Agents [12.9624447364193]
  Self-evolving agent frameworks provide no formal guarantees of safety or correctness.<n>We formulate agentic code generation as a constrained learning problem, combining hard formal specifications with soft objectives capturing task utility.<n>We present SEVerA, a three-stage framework: Search synthesizes candidate parametric programs containing FGGM calls; Verification proves correctness with respect to hard constraints for all parameter values, reducing the problem to unconstrained learning.
  arXiv  Detail & Related papers  (2026-03-26T07:32:20Z)
• DRAFTO: Decoupled Reduced-space and Adaptive Feasibility-repair Trajectory Optimization for Robotic Manipulators [4.0407133618465005]
  This paper introduces a new algorithm for trajectory optimization, Decoupled Reduced-space and Adaptive Feasibility-repair Trajectory Optimization (DRAFTO)<n>To reduce the number of repeated constrained optimizations while handling joint-limit feasibility, the optimization is decoupled into a reduced-space Gauss-Newton (GN) descent.<n>The results of our benchmark tests against optimization-based planners, such as CHOMP, TrajOpt, GPMP2, and FACTO, validate the high efficiency and reliability across diverse scenarios and tasks.
  arXiv  Detail & Related papers  (2026-03-10T20:24:42Z)
• HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders [13.73393292649997]
  We propose HiSAC, an efficient framework for personalized sequence modeling.<n>HiSAC encodes interactions into multi-level semantic IDs and constructs a global hierarchical codebook.<n>A hierarchical voting mechanism sparsely activates personalized interest-agents as fine-grained preference centers.<n>Soft-Routing Attention aggregates historical signals in semantic space, weighting by similarity to minimize quantization error.
  arXiv  Detail & Related papers  (2026-02-24T15:28:58Z)
• CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning [4.765206163164323]
  CLEANER exploits intrinsic self-correction capabilities to eliminate error-contaminated context during data collection.<n>Similarity-Aware Adaptive Rollback mechanism autonomously constructs clean, purified trajectories.<n>Results show average accuracy gains of 6%, 3%, and 5% over baselines.
  arXiv  Detail & Related papers  (2026-01-21T16:14:30Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models [45.72323731094864]
  Low-Rank Adaptation (LoRA) emerges as a popular parameter-efficient fine-tuning (PEFT) method. In this work, we study the enhancement of LoRA training by introducing an $r times r$ preconditioner in each gradient step.
  arXiv  Detail & Related papers  (2024-02-04T05:05:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.