LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

• URL: http://arxiv.org/abs/2608.01964v1
• Date: Mon, 03 Aug 2026 09:32:21 GMT
• Title: LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
• Abstract summary: LongHorizon-Harness maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment.<n>A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops.
• Score: 22.929016343684395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within a growing context, making the state difficult to track and allowing incorrect self-assessments to propagate into later decisions. We reformulate long-horizon execution as a task-state management problem and propose LongHorizon-Harness, which maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment. Its Manage-Execute-Audit(MEA) loop uses a manager to maintain the task state and determine the next subtask, a fresh-context executor to perform it, and a read-only auditor to verify the resulting environment state before the next round. A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops. LongHorizon-Harness improves Qwen~3.7-Plus from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench~2.1, and from 2.8% to 8.3% on OSWorld~2.0. It also raises Claude Opus~4.7 from 20.0% to 34.3% on an OSWorld2.0 subset, demonstrating consistent gains across models, harnesses, and interaction domains.

Related papers

• OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents [52.15259607762389]
  We present OneDayAgent, a long-horizon harness for autonomous agents.<n>OneDayAgent turns an open-ended request into a managed execution process.<n>We evaluate OneDayAgent on AgentIF-OneDay across 104 tasks.
  arXiv  Detail & Related papers  (2026-08-04T17:55:41Z)
• StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure [29.25570731801935]
  StructAgent is a state-centered framework for maintaining compact, verifiable task progress.<n>It enables explicit progress checkpointing, evidence-driven task completion, targeted failure recovery, and tool-supported execution.<n>It generalizes beyond desktop environments to Minecraft, demonstrating the generality of our design.
  arXiv  Detail & Related papers  (2026-07-13T10:48:25Z)
• Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading [46.911171278867585]
  Long-Horizon-Terminal-Bench is a terminal benchmark of 46 long-horizon tasks spanning nine categories.<n>Each task follows a Terminal-Bench-style setup with a reference solution or simulation engine, but is further graded into fine-grained sub-tasks.<n>We evaluate 15 frontier models and find that agents consume on average 9.9M tokens per task, with roughly 231 episodes and 85.3 minutes of execution time per run.
  arXiv  Detail & Related papers  (2026-07-09T21:56:37Z)
• OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks [98.07146747892239]
  OSWorld 2.0 is a benchmark of 108 long-horizon computer-use across everyday and professional tasks.<n>Each task represents a realistic end-to-end workflow that takes human users a median of about 1.6 hours to complete.
  arXiv  Detail & Related papers  (2026-06-28T17:59:17Z)
• WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [88.10947115397971]
  This work presents WildClawBench, a native-runtime benchmark of 60 human-authored, bilingual, multimodal tasks spanning six thematic categories.<n>Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls.<n> Grading is hybrid, combining deterministic rule-based checks, environment-state auditing of side effects, and an LLM/VLM judge for semantic verification.
  arXiv  Detail & Related papers  (2026-05-11T17:49:43Z)
• ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents [77.22389710754452]
  We introduce bench, a benchmark for coworker agents built around multi-turn multi-day tasks.<n>The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services.<n>The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common.
  arXiv  Detail & Related papers  (2026-04-26T16:05:02Z)
• Structural Verification for Reliable EDA Code Generation without Tool-in-the-Loop Debugging [0.6843491191969066]
  We propose to eliminate tool-in-the-loop debug by enforcing structural correctness prior to execution.<n>On single-step tasks, our method improves pass rate from 73.0% (LLM+RAG) and 76.4% (tool-in-loop) to 82.5%.<n>On multi-step tasks, pass rate improves from 30.0% to 70.0%, and further to 84.0% with trajectory-level reflection.
  arXiv  Detail & Related papers  (2026-04-20T20:58:52Z)
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces [126.23612941699565]
  Terminal-Bench 2.0 is a benchmark composed of 89 tasks in computer terminal environments inspired by problems from real world.<n>We show that frontier models and agents score less than 65% on the benchmark.<n>We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/.
  arXiv  Detail & Related papers  (2026-01-17T01:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.