Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents

• URL: http://arxiv.org/abs/2605.23574v1
• Date: Fri, 22 May 2026 12:44:01 GMT
• Title: Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents
• Abstract summary: Long-horizon language agents can make many plausible local tool calls yet fail to persist until a requested count is actually complete.<n>We study this gap as Quantitative Goal Persistence (QGP): whether an agent keeps working until an external verifier confirms enough distinct valid items.<n>PushBench turns this into a benchmark for repository-artifact collection and verifier-backed work units.
• Score: 7.058932007120959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon language agents can make many plausible local tool calls yet fail to persist until a requested count is actually complete. We study this gap as Quantitative Goal Persistence (QGP): whether an agent keeps working until an external verifier confirms enough distinct valid items. PushBench turns this into a benchmark for repository-artifact collection and verifier-backed work units, so repeated work, duplicate submissions, false completion, and progress drift are measured directly rather than hidden behind a final success flag. In matched controller comparisons, a state-tracking retrieval controller reaches 69-78% success while eliminating duplicate submissions, and a backlog-tracking work-unit controller reaches 25-50% success in settings where standard and completion-gated controllers complete no task instances. Black-box frontier-agent evaluations with Claude Code (Sonnet 4.6) and Codex CLI (gpt-5.4) solve many 50-artifact tasks but drop to 3 out of 9 successes per condition at 100 artifacts. The results show that quantitative goals stress a different reliability requirement from local task competence: agents must maintain verified progress and stop only when the requested work is complete.

Related papers

• The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution? [9.023267560784042]
  Recent large language models can emit task-progress signals that agent frameworks use to decide whether a task should continue or stop.<n>We evaluate this ability on the public benchmark $2$-bench and on StageIF, a controlled testbed in which reporting checkpoints are placed across the task's lifecycle.<n>We find that reporting reliability depends on the stage a task has reached, and that almost every deployed model we test is reliable at some stages and unreliable at others.
  arXiv  Detail & Related papers  (2026-09-08T11:25:31Z)
• TraceGrant: A Contract-Governed Security Framework for the Task-Effect Lifecycle of Networked LLM Agents [7.080139152914316]
  We present TraceGrant, a security framework that governs the task-effect lifecycle of networked LLM agents.<n>Before execution, TraceGrant establishes a task-effect boundary from the trusted user request.<n>During execution, admitted evidence can instantiate only authority already established by the Contract.<n>After execution, task completion is verified against actual tool results.
  arXiv  Detail & Related papers  (2026-08-21T14:05:42Z)
• One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows [2.281697288703173]
  Thinkingbox is a sandbox for tool agent-user interaction.<n>It provides isolated MCP-compatible tool sessions, complete execution traces, and outcome evaluation over terminal backend.<n> Thinkingboxbench contains 507 benchmarks and state-conditioned benchmarks across numerous scenarios.
  arXiv  Detail & Related papers  (2026-08-20T07:37:57Z)
• OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents [52.15259607762389]
  We present OneDayAgent, a long-horizon harness for autonomous agents.<n>OneDayAgent turns an open-ended request into a managed execution process.<n>We evaluate OneDayAgent on AgentIF-OneDay across 104 tasks.
  arXiv  Detail & Related papers  (2026-08-04T17:55:41Z)
• Turning Interaction History into Execution State: A Runtime Layer for Long-Horizon Coding Agents [48.967927222079965]
  Ledger is a runtime layer that distills an agent's completed interactions into an explicit execution state.<n> Ledger keeps this state in an online execution ledger and applies it at two boundaries of every step.<n>Across all 500 SWE-bench Verified instances, Ledger raises Pass@1 from 56.2% to 64.2% with GPT-5 mini and from 75.8% to 81.0% with MiniMax M2.5.
  arXiv  Detail & Related papers  (2026-08-01T18:25:00Z)
• RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications [0.0]
  RuBench 1.0 is a benchmark of 25 tasks mined from recent fix commits in five live open-source repositories.<n>Each task is specified in Russian, written from scratch in the style of an actual customer request, not translated.<n>All 25 fix commits postdate the training-data cutoffs of every evaluated model.
  arXiv  Detail & Related papers  (2026-07-07T15:41:22Z)
• Verify-Gated Completion as Admission Control in a Governed Multi-Agent Runtime: A Bounded Architecture Case Study [0.6875312133832079]
  We study verify-gated completion as an admission-control pattern for governed multi-agent runtimes.<n>We show that under observed conditions, a read-only verify gate plus packetized admission records made completion decisions inspectable and fail-closed.
  arXiv  Detail & Related papers  (2026-05-18T07:52:13Z)
• Good to Go: The LOOP Skill Engine That Hits 99% Success and Slashes Token Usage by 99% via One-Shot Recording and Deterministic Replay [14.181844060152367]
  We present the LOOP SKILL ENGINE, a system that achieves a combined 99% success rate and 99% token reduction for periodic agent tasks.<n>A deterministic execution plan captures the task's functional intent while parameterizing time-dependent and result-dependent variables.<n>Across a benchmark of periodic agent tasks spanning intervals from 5 minutes to 24 hours, the Loop Skill Engine reduces monthly token consumption by 93.3%--99.98% and cuts execution latency by 8.7x while eliminating output non-determinism.
  arXiv  Detail & Related papers  (2026-05-14T01:05:35Z)
• WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [88.10947115397971]
  This work presents WildClawBench, a native-runtime benchmark of 60 human-authored, bilingual, multimodal tasks spanning six thematic categories.<n>Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls.<n> Grading is hybrid, combining deterministic rule-based checks, environment-state auditing of side effects, and an LLM/VLM judge for semantic verification.
  arXiv  Detail & Related papers  (2026-05-11T17:49:43Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation [98.38575149237442]
  VLAA-GUI is a modular GUI framework built around three integrated components.<n>A mandatory Completeness Verifier enforces UI-observable success criteria and verification at every finish step.<n>A mandatory Loop Breaker provides multi-tier switching interaction mode after repeated failures.
  arXiv  Detail & Related papers  (2026-04-23T07:42:37Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging [41.754784344572286]
  We release GitTaskBench, a benchmark for evaluating code agents in real-world scenarios.<n>Each task pairs a relevant repository with an automated, human-curated evaluation harness.<n>We also propose the alpha-value metric to quantify the economic benefit of agent performance.
  arXiv  Detail & Related papers  (2025-08-26T12:48:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.