Belayer: Efficient Fault Tolerance for LLM Agentic RL Training

• URL: http://arxiv.org/abs/2608.14635v2
• Date: Tue, 18 Aug 2026 03:09:25 GMT
• Title: Belayer: Efficient Fault Tolerance for LLM Agentic RL Training
• Abstract summary: Belayer is an efficient fault-tolerant system for large language model (LLM) agentic RL training.<n>It handles failures in both rollout engines and environment execution while targeting low failure-free overhead.<n>For environment failures, Belayer introduces full checkpoint installation and full restore to jointly capture and restore container file-system and state.
• Score: 28.236424334567133
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language model (LLM) agents are increasingly trained with reinforcement learning in long-horizon, sandboxed environments. Unlike conventional RL, agentic RL couples GPU-intensive rollout engines with stateful environment containers whose actions may produce visible side effects, such as file edits, command execution, and dependency installation. A single trajectory can span many rounds of gen- eration and environment interaction, so a component failure can discard completed work or expose the model to an environment state that is inconsistent with its context. However, existing systems lack efficient and correct recovery mechanisms for this distributed execution model. This paper presents Belayer, an efficient fault-tolerant system for LLM agentic RL training. Belayer handles failures in both rollout engines and environment execution while targeting low failure-free overhead. For scoped worker-local rollout failures, Belayer equips each pre-initialized shadow worker with a selective GPU-state reuse protocol that retains independently owned weights and raw KV-arena allocations after owner and GPU health checks, reinitializes worker-local state, and rebuilds request-specific KV contents from logged token prefixes. For environment failures, Belayer introduces full checkpoint and full restore to jointly capture and restore container file-system and runtime state, and coordinates the recovered environment with the LLM context to preserve prefix consistency. An adaptive policy opportunistically overlaps full-state checkpointing with natural LLM inference bubbles when the predicted interval is long enough. Empirical results show low measured overhead during failure-free training, a worker-recovery-time reduction of up to 42 times faster compared with a full engine cold start, and 1.5 to 3.5 times faster recovery from environment failures.

Related papers

• MCP-Universe RL: A Framework for Training MCP Tool-Use Agents via Reinforcement Learning [54.082979654934]
  Reinforcement learning (RL) has become an effective way to improve the tool-use ability of large language models (LLMs)<n>Most existing RL frameworks stop at the policy update.<n>We present MCP-Universe RL (MCP-U RL), an open-source framework that takes over both.
  arXiv  Detail & Related papers  (2026-08-23T01:49:04Z)
• REPREC: Representation Driven Parameter-Efficient Recommendation System [2.7992557071345305]
  Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task.<n>Previous work has improved personalization by incorporating collaborative and sequential signals through input conditioning or LLM fine-tuning.<n>We propose REPREC, a lightweight framework that reformulates LLM-based sequential recommendation through lightweight user representation alignment.
  arXiv  Detail & Related papers  (2026-07-24T22:45:33Z)
• PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR [33.08614139144317]
  We present PlexRL, a cluster-level runtime for multiplexing unified LLM services across RLVR jobs.<n>Our implementation and evaluations demonstrate that PlexRL significantly improves effective cluster capacity and reduces user GPU hour cost by maximum 37.58%.
  arXiv  Detail & Related papers  (2026-05-20T07:55:06Z)
• EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL [54.09410318521061]
  We introduce EnvFactory, a fully automated framework for Agentic Reinforcement Learning (Agentic RL) training.<n>EnvFactory autonomously explores and stateful, executable tool environments from authentic resources.<n>It synthesizes natural multi-turn trajectories through topology-aware sampling and calibrated refinement.<n>It achieves superior training efficiency and downstream performance, improving Qwen3-series models by up to +15% on BFCLv3, +8.6% on MCP-Atlas, and +6% on conversational benchmarks including $2$-Bench and VitaBench.
  arXiv  Detail & Related papers  (2026-05-18T17:37:40Z)
• ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload [18.933354857797056]
  Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare.<n>We propose ReCoVer, a resilient pre-training system that upholds a single invariant.<n>For comparison with checkpoint-and-restart baselines, ReCoVer demonstrates $2.23times$ higher effective throughput after successive failures.
  arXiv  Detail & Related papers  (2026-05-11T20:28:31Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents [70.68963723787424]
  TermiGen is an end-to-end pipeline for synthesizing verifiable environments and resilient expert trajectories.<n>Our TermiGen-Qwen2.5-Coder-32B achieves a 31.3% pass rate on TerminalBench.
  arXiv  Detail & Related papers  (2026-02-06T23:56:50Z)
• ELLA: Efficient Lifelong Learning for Adapters in Large Language Models [12.489255789379817]
  Large Language Models (LLMs) suffer severe catastrophic forgetting when adapted sequentially to new tasks in a continual learning setting.<n>We introduce ELLA, a training framework built on the principle of selective subspace de-correlation.<n>ELLA explicitly characterizes the structure of past updates and penalizes alignments along their high-energy, task-specific directions.<n>It achieves state-of-the-art CL performance on three popular benchmarks, with relative accuracy gains of up to $9.6%$ and a $35times$ smaller memory footprint.
  arXiv  Detail & Related papers  (2026-01-05T15:58:08Z)
• MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment [14.392166280035122]
  Large language models (LLMs) deployed on edge servers are increasingly used in latency-sensitive applications such as personalized assistants, recommendation, and content moderation.<n>Existing retraining strategies either delay model updates, over-commit resources to retraining, or overlook iteration-level retraining granularity.<n>We propose MACE, a hybrid LLM system that colocates concurrent inference (prefill, decode) and fine-tuning, with intelligent memory management to maximize task performance while promising inference throughput.
  arXiv  Detail & Related papers  (2025-09-28T18:45:28Z)
• Reinforcement Learning for Machine Learning Engineering Agents [52.03168614623642]
  We show that agents backed by weaker models that improve via reinforcement learning can outperform agents backed by much larger, but static models.<n>We propose duration- aware gradient updates in a distributed asynchronous RL framework to amplify high-cost but high-reward actions.<n>We also propose environment instrumentation to offer partial credit, distinguishing almost-correct programs from those that fail early.
  arXiv  Detail & Related papers  (2025-09-01T18:04:10Z)
• StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation [55.75008325187133]
  Reinforcement learning (RL) has become the core post-training technique for large language models (LLMs)<n>StreamRL is designed with disaggregation from first principles to address two types of performance bottlenecks.<n> Experiments show that StreamRL improves throughput by up to 2.66x compared to existing state-of-the-art systems.
  arXiv  Detail & Related papers  (2025-04-22T14:19:06Z)
• EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation [84.70637613266835]
  EoRA is a fine-tuning-free method that augments compressed Large Language Models with low-rank matrices.<n>EoRA consistently outperforms prior training-free low rank methods in recovering the accuracy of compressed LLMs.
  arXiv  Detail & Related papers  (2024-10-28T17:59:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.