Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents

• URL: http://arxiv.org/abs/2609.11677v1
• Date: Thu, 10 Sep 2026 15:09:08 GMT
• Title: Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
• Abstract summary: Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents.<n>Existing harness evolution methods rely on iterative search, repeatedly evaluating and revising candidate harnesses.<n>We propose Ecdysis, an efficient and effective framework that distinguishes model-specific accommodation from systematic harness deficiencies.
• Score: 25.41577484327689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing harness evolution methods typically rely on iterative search, repeatedly evaluating and revising candidate harnesses based on execution feedback from task instances. While this paradigm enables continuous harness optimization, it incurs substantial time overhead due to repeated agent executions and code modifications, and may overfit to observed tasks and specific failure patterns, resulting in degraded generalization to unseen tasks. We identify the lack of principled failure diagnosis as a key bottleneck in harness evolution: an observed failure can reflect either model-specific deficiencies or systematic harness deficiencies, and directly optimizing against individual failures can lead to unnecessary model-specific accommodation. We therefore propose Ecdysis, an efficient and effective framework that distinguishes model-specific accommodation from harness-level repair and biases adaptation toward systematic harness deficiencies by identifying recurring cross-task failure patterns. Ecdysis adopts a batch-level cross-instance failure aggregation paradigm to jointly analyze failure evidence from multiple task instances and further introduces Failure-Driven Collaborative Refinement to diagnose failure causes and iteratively refine harness modification specifications. By combining cross-instance failure analysis with multi-role diagnosis, Ecdysis enables more effective harness evolution with lower training time. Experiments show that Ecdysis achieves up to a 1.84x speedup in harness training compared with existing harness evolution methods, while improving the reasoning accuracy of the resulting harnesses by 18.56%.

Related papers

• ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement [46.36989489677189]
  We propose a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution.<n> ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies.<n>It decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection.
  arXiv  Detail & Related papers  (2026-09-14T00:10:45Z)
• Evo-Bench: Can Language Models Improve Agent Harness? [44.72089415085898]
  Evo-Bench is the first benchmark designed to evaluate models' intrinsic harness-evolving capabilities across Search, Office, and General agent domains.<n>Top models achieve massive absolute gains reaching 16.6 points, closely approaching state-of-the-art human-engineered baselines.
  arXiv  Detail & Related papers  (2026-08-10T03:49:28Z)
• Recursive Harness Self-Improvement [28.432258880153125]
  Under model--harness co-evolution, harnesses are data-generating components whose execution traces can shape future foundation models.<n>We introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop.<n>RHI iteratively refines it using pairwise feedback over its own revision history.
  arXiv  Detail & Related papers  (2026-07-17T00:21:19Z)
• Rethinking the Evaluation of Harness Evolution for Agents [83.07258924910069]
  Existing harness evolution methods use unit test cases to search for harness configurations and then report final performance on the same public benchmark.<n>This protocol raises two fundamental concerns.<n>As in agentic test-time scaling, it should therefore be compared with simple task-level search baselines under matched feedback and inference budgets.
  arXiv  Detail & Related papers  (2026-07-14T00:18:42Z)
• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• Harnesses for Inference-Time Alignment over Execution Trajectories [13.182534464050695]
  We study harness design through the lens of inference-time trajectory alignment.<n>This decomposition allows us to quantify how workflow, retry budgets, and guidance-induced action reweighting shape the performance limits of harness design.<n>Inspired by the theory, we show that effective harnesses can be partial: specifying only the initial steps and leaving the remaining execution to agent can achieve higher pass rate.
  arXiv  Detail & Related papers  (2026-05-15T12:47:13Z)
• On Predicting the Post-training Potential of Pre-trained LLMs [60.07459271263409]
  We introduce a new task of predicting post-training potential - forecasting a base model's performance before post-training.<n>We propose RuDE, a unified framework that bypasses the generation gap of base models by leveraging response discrimination.<n>Experiments demonstrate a correlation greater than 90% with post-training performance.
  arXiv  Detail & Related papers  (2026-05-12T11:33:49Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning [0.3823356975862005]
  We propose failure- conditioning, a simple and effective method for learning from saturated problems.<n>We observe that failure-prone conditioning yields performance gains matching those of training on medium-difficulty problems.<n>Our results suggest that failure- conditioning offers an effective pathway to extend RLVR training on saturated problems.
  arXiv  Detail & Related papers  (2026-01-28T18:29:21Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.