On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length

• URL: http://arxiv.org/abs/2605.02572v1
• Date: Mon, 04 May 2026 13:25:05 GMT
• Title: On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
• Abstract summary: We study the role of task horizon length in shaping training dynamics.<n>We find that increasing horizon length alone constitutes a training bottleneck.<n>We demonstrate that horizon reduction is a key principle to address this limitation.
• Score: 62.99653688239127
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) have shown promise as interactive agents that solve tasks through extended sequences of environment interactions. While prior work has primarily focused on system-level optimizations or algorithmic improvements, the role of task horizon length in shaping training dynamics remains poorly understood. In this work, we present a systematic empirical study that examines horizon length through controlled task constructions. Specifically, we construct controlled tasks in which agents face identical decision rules and reasoning structures, but differ only in the length of action sequences required for successful completion. Our results reveal that increasing horizon length alone constitutes a training bottleneck, inducing severe training instability driven by exploration difficulties and credit assignment challenges. We demonstrate that horizon reduction is a key principle to address this limitation, stabilizing training and achieving better performance in long-horizon tasks. Moreover, we find that horizon reduction is related to stronger generalization across horizon lengths: models trained under reduced horizons generalize more effectively to longer-horizon variants at inference time, a phenomenon we refer to as horizon generalization.

Related papers

• HIPIF: Hierarchical Planning and Information Folding for Long-Horizon LLM Agent Learning [51.95194664412812]
  Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents across a wide range of tasks.<n>We propose Hierarchical Planning and Information Folding (HIPIF) for long-horizon LLM agent learning.<n>HIPIF trains the agent end-to-end to organize long-horizon execution around explicit subgoals while folding completed subgoal histories to reduce long-context interference.
  arXiv  Detail & Related papers  (2026-06-09T07:35:14Z)
• When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution [17.020655152761346]
  LongAct is a benchmark designed to evaluate planning-level autonomy in long-horizon household tasks.<n>HoloMind is a VLM-driven agent with a DAG-based long-horizon hierarchical planner.
  arXiv  Detail & Related papers  (2026-05-14T07:47:53Z)
• The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break [69.91172974854422]
  HORIZON is a cross-domain diagnostic benchmark for analyzing long-horizon failure behaviors in large language model (LLM) agents.<n>We evaluate state-of-the-art (SOTA) agents from multiple model families to study horizon-dependent degradation patterns.<n>Our findings offer an initial methodological step toward systematic, cross-domain analysis of long-horizon agent failures.
  arXiv  Detail & Related papers  (2026-04-13T19:11:42Z)
• LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces [65.11019654023978]
  LongCLI-Bench is a benchmark designed to evaluate agentic capabilities across long-horizon, realistic tasks.<n>We curated 20 high-quality, long-horizon tasks from over 1,000 computer science assignments and real-world tasks.<n>Experiments reveal that even state-of-the-art agents achieve pass rates below 20% in LongCLI-Bench.
  arXiv  Detail & Related papers  (2026-02-15T23:12:57Z)
• h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning [22.930073904843212]
  Large language models excel at short-horizon reasoning tasks, but performance drops as reasoning horizon lengths increase.<n>Existing approaches to combat this rely on inference-time scaffolding or costly step-level supervision.<n>We introduce a scalable method to bootstrap long-horizon reasoning capabilities using only existing, abundant short-horizon data.
  arXiv  Detail & Related papers  (2025-10-08T17:58:41Z)
• UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarios [63.67884284105684]
  We introduce textbfUltraHorizon, a novel benchmark that measures the foundational capabilities essential for complex real-world challenges.<n>Agents are designed in long-horizon discovery tasks where they must iteratively uncover hidden rules.<n>Our experiments reveal that LLM-agents consistently underperform in these settings, whereas human participants achieve higher scores.
  arXiv  Detail & Related papers  (2025-09-26T02:04:00Z)
• State-Covering Trajectory Stitching for Diffusion Planners [29.89423911968709]
  State-Covering Trajectory Stitching (SCoTS) is a reward-free trajectory augmentation method that stitches together short trajectory segments.<n>We demonstrate that SCoTS significantly improves the performance and generalization capabilities of diffusion planners on offline goal-conditioned benchmarks.
  arXiv  Detail & Related papers  (2025-06-01T08:32:22Z)
• Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning [32.260964481673085]
  Large language models (LLMs) struggle with long-horizon decision-making tasks due to deficient exploration and long-term credit assignment.<n>We propose an innovative framework that introduces a parameter-efficient and generally applicable hierarchy to LLM policies.<n>We develop a scheme where the low-level controller is supervised with abstract, step-by-step plans that are learned and instructed by the high-level policy.
  arXiv  Detail & Related papers  (2025-05-26T09:43:40Z)
• Open-World Reinforcement Learning over Long Short-Term Imagination [91.28998327423295]
  Training visual reinforcement learning agents in a high-dimensional open world presents significant challenges.<n>We present LS-Imagine, which extends the imagination horizon within a limited number of state transition steps.<n>Our method demonstrates significant improvements over state-of-the-art techniques in MineDojo.
  arXiv  Detail & Related papers  (2024-10-04T17:17:30Z)
• Efficient Learning of High Level Plans from Play [57.29562823883257]
  We present Efficient Learning of High-Level Plans from Play (ELF-P), a framework for robotic learning that bridges motion planning and deep RL. We demonstrate that ELF-P has significantly better sample efficiency than relevant baselines over multiple realistic manipulation tasks.
  arXiv  Detail & Related papers  (2023-03-16T20:09:47Z)
• Challenges to Solving Combinatorially Hard Long-Horizon Deep RL Tasks [25.37125069796657]
  We propose a set of tasks which admit many distinct solutions at the high-level, but require reasoning about states and rewards thousands of steps into the future for the best performance. We find that standard RL methods often neglect long-term effects due to discounting, while general-purpose hierarchical RL approaches struggle unless additional abstract domain knowledge can be exploited.
  arXiv  Detail & Related papers  (2022-06-03T20:38:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.