CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

• URL: http://arxiv.org/abs/2607.05378v1
• Date: Mon, 06 Jul 2026 17:55:12 GMT
• Title: CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents
• Abstract summary: We propose CompactionRL, a reinforcement learning strategy to train long-horizon agentic LLMs with context compaction.<n>We train CompactionRL on top of open models and observe consistent performance gains on agentic coding tasks.<n>Built upon GLM-4.7-Flash (30B-A3B), CompactionRL improves Pass@1 by 5.5 and 6.8 points, reaching 56.4% on SWE-bench Verified and 20.2% on Terminal-Bench 2.0.
• Score: 46.180980184707416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon agentic LLMs are increasingly limited by finite context windows, as extended interaction trajectories can exceed the maximum context length before a task is completed. Context compaction offers a natural solution by summarizing previous interaction states and continuing the rollout under a compressed context, but incorporating compaction into reinforcement learning remains underexplored. We propose CompactionRL, a reinforcement learning strategy to train long-horizon agentic LLMs with context compaction. Our approach jointly optimizes task execution and summary generation with token-level loss normalization and cross-trajectory generalized advantage estimation. This design enables the LLM agents to learn from compacted long-horizon trajectories. We train CompactionRL on top of open models and observe consistent performance gains on agentic coding tasks. CompactionRL enables the open GLM-4.5-Air model (106B-A30B) to achieve Pass@1 scores of 66.8% on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, with absolute gains of 7.0 and 3.1 points, respectively. Built upon GLM-4.7-Flash (30B-A3B), CompactionRL improves Pass@1 by 5.5 and 6.8 points, reaching 56.0% on SWE-bench Verified and 20.2% on Terminal-Bench 2.0, respectively. CompactionRL is thus deployed in the RL pipeline for training the open GLM-5.2 model (750B-A40B).

Related papers

• LAPF: LLM-Agent-Based Path Finder Using the UAVScenes Dataset [8.27333944529512]
  This paper proposes an LLM-Agent-Based Path Finder (LAPF) framework for autonomous UAV navigation in town-scale outdoor environments.<n>LAPF extends LLM-assisted navigation by integrating perception, memory, planning, and action modules into a closed-loop cognitive architecture.<n>LAPF achieves mean path lengths of 512.83 m and 506.37 m, compared to the straight-line optimum of 497.33 m.
  arXiv  Detail & Related papers  (2026-08-15T11:28:21Z)
• Mach-Mind-4-Flash Technical Report [0.0]
  Mach-Mind-4-Flash is a Mixture-of-Experts (MoE) agentic model with 3B activated parameters.<n>Post-training optimization alone without scaling pre-training compute, the model achieves performance on par with or surpassing that of 100B- parameter-class models.
  arXiv  Detail & Related papers  (2026-07-10T12:57:08Z)
• Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training? [23.212983812736343]
  Agent2 RL-Bench is a benchmark for evaluating agentic RL post-training.<n>We find that supervised pipelines dominate agent-driven post-training under fixed budgets.
  arXiv  Detail & Related papers  (2026-04-12T09:35:27Z)
• Language-based Trial and Error Falls Behind in the Era of Experience [50.503828360874536]
  Large Language Models (LLMs) excel in language-based agentic tasks, but their applicability to unseen, nonlinguistic environments remains limited.<n>In this work, we demonstrate the primary bottleneck is the prohibitive cost of exploration.<n>We propose SCOUT, a novel framework that decouples exploration from semantic exploitation.
  arXiv  Detail & Related papers  (2026-01-29T14:08:41Z)
• QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs [80.76334908639745]
  We propose QeRL, a Quantization-enhanced Reinforcement Learning framework for large language models (LLMs)<n>QeRL addresses issues by combining NVFP4 quantization with Low-Rank Adaptation (LoRA)<n>Experiments demonstrate that QeRL delivers over 1.5 times speedup in the rollout phase.
  arXiv  Detail & Related papers  (2025-10-13T17:55:09Z)
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models [194.64264251080454]
  We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters.<n>Through multi-stage training on 23T tokens and comprehensive post-training with expert model iteration and reinforcement learning, GLM-4.5 achieves strong performance across agentic, reasoning, and coding tasks.<n>We release both GLM-4.5 (355B parameters) and a compact version, GLM-4.5-Air (106B parameters), to advance research in reasoning and agentic AI systems.
  arXiv  Detail & Related papers  (2025-08-08T17:21:06Z)
• RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization [111.1749164063616]
  We propose RL-PLUS, a novel hybrid-policy optimization approach for Large Language Models (LLMs)<n> RL-PLUS synergizes internal exploitation with external data to achieve stronger reasoning capabilities and surpass the boundaries of base models.<n>We provide both theoretical analysis and extensive experiments to demonstrate the superiority and generalizability of our approach.
  arXiv  Detail & Related papers  (2025-07-31T23:55:29Z)
• L0: Reinforcement Learning to Become General Agents [9.210885263681119]
  We introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents.<n> NB-Agent, the agent scaffold within L0, operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL)<n>Experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR)
  arXiv  Detail & Related papers  (2025-06-30T09:44:32Z)
• Controlling Large Language Model with Latent Actions [27.0292050543406]
  Adapting Large Language Models to downstream tasks using Reinforcement Learning has proven to be an effective approach.<n>This paper studies learning a compact latent action space to enhance the controllability and exploration of RL for LLMs.<n>We propose Controlling Large Language Models with Latent Actions (CoLA), a framework that integrates a latent action space into pre-trained LLMs.
  arXiv  Detail & Related papers  (2025-03-27T11:25:22Z)
• LLMStinger: Jailbreaking LLMs using RL fine-tuned LLMs [13.36946005380889]
  We introduce LLMStinger, a novel approach that leverages Large Language Models (LLMs) to automatically generate adversarial suffixes for jailbreak attacks. Our method significantly outperforms existing red-teaming approaches, achieving a +57.2% improvement in Attack Success Rate (ASR) on LLaMA2-7B-chat and a +50.3% increase on Claude 2.
  arXiv  Detail & Related papers  (2024-11-13T18:44:30Z)
• Why Does the Effective Context Length of LLMs Fall Short? [68.34573617977013]
  In this work, we introduce ShifTed Rotray position embeddING (STRING) STRING shifts well-trained positions to overwrite the original ineffective positions during inference, enhancing performance within their existing training lengths. Experimental results show that STRING dramatically improves the performance of the latest large-scale models.
  arXiv  Detail & Related papers  (2024-10-24T13:51:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.